TW202407330A - Automatic defect classification device - Google Patents

Abstract

Provided is an automatic defect classification device that facilitates correction of errors in classification categories assigned in advance by a user and enables improvement in the accuracy of automatic defect classification, even when there are many defect images constituting training data. The automatic defect classification device is for generating, by using training data in which classification categories clearly indicating associations of a plurality of defect images with defect classifications have been assigned to the defect images in advance by the user, classifiers corresponding to the defect images and for, by using the classifiers, automatically assigning the classification categories to defect images of which classification categories are unknown. The automatic defect classification device comprises: a training data input unit that receives input of training data; a processing unit that performs a predetermined process on the plurality of defect images; and a display unit that displays the defect images. The processing unit causes the display unit to perform display in a changed display format of defect portions of the defect images in accordance with the classification categories which have been assigned by the classifiers and with which the defect images are associated.

Description

Automatic defect classification device

The present invention relates to an automatic defect classification device that uses teaching materials including a plurality of defect images assigned classification categories by a user to create a classification standard, and automatically determines classification categories for defect images with unknown classification categories based on the classification standard.

For example, a semiconductor device is layered on a semiconductor wafer to form a plurality of semiconductor device circuits (i.e., a repeating appearance pattern of the device wafer), which are then singulated into individual chip components, and the wafer components are packaged as electronic components. Shipped as a separate unit or assembled into electrical products.

Furthermore, before being singulated into individual wafer parts, one semiconductor wafer is kept and film formation, exposure, development, etching, smoothing, etc. are repeatedly performed, and defects of the device wafer formed on the wafer are detected during the process. Detection and classification of defect types (for example, Patent Document 1). [Prior technical literature] [Patent Document]

[Patent Document 1] Japanese Patent Application Publication No. 2007-071586

[Problem to be solved by the invention]

In order to automatically determine the classification category for defect images whose classification categories are unknown, teaching materials are used to create classification standards. However, the plurality of defect images constituting the teaching materials have already been assigned classification categories by the user. Therefore, if there are errors in the classification categories assigned by users in the teaching materials, the classification accuracy will be reduced.

On the other hand, the greater the number of plural defect images that make up teaching materials, the higher the accuracy of automatic classification. However, since it involves visual inspection, it becomes increasingly difficult to detect errors as the number increases. , therefore, it is difficult to improve the classification accuracy.

Therefore, the present invention was completed in view of the above-mentioned problems. An object of the present invention is to provide an automatic defect classification device that can easily correct errors in classification categories assigned by users in advance even if there are many defect images constituting teaching materials, thereby improving the accuracy of automatic defect classification. [Technical means to solve problems]

In order to solve the above problems, one aspect of the present invention is an automatic defect classification device. Using teaching materials that have been assigned by the user in advance to each of the plurality of defect images, a classification category that clearly indicates which defect category the defect image belongs to is used to generate a classifier for the defect image, and the classifier is used to classify the unknown classification categories Defect images are automatically assigned classification categories; the automatic defect classification device has: Teaching material input department, which inputs teaching materials; A processing unit that performs prescribed processing on a plurality of defective images; and a display portion that displays an image of the defect; and The processing unit changes and displays the display format of the defective part of the defective image in the display unit according to the classification category to which the defective image belongs given by the classifier.

Furthermore, in order to solve the above problem, another aspect of the present invention is an automatic defect classification device. Using teaching materials that have been assigned by the user in advance to each of the plurality of defect images, a classification category that clearly indicates which defect category the defect image belongs to is used to generate a classifier for the defect image, and the classifier is used to classify the unknown classification categories Defect images are automatically assigned classification categories; the automatic defect classification device has: Teaching material input department, which inputs teaching materials; A processing unit that performs prescribed processing on a plurality of defective images; and a display portion that displays an image of the defect; and The processing unit changes and displays the display form of the defective part of the defective image in the display unit according to the classification accuracy of the classification category to which the defective image belongs given by the classifier. [Effects of the invention]

Even if there are many defect images constituting the teaching materials, it is easy to correct errors in the classification categories assigned by the user in advance, which can improve the accuracy of automatic defect classification.

Hereinafter, embodiments for carrying out the present invention will be described using drawings.

FIG. 1 is an image diagram showing an example of a defect image in an embodiment embodying the present invention. Defect images P1 to P4 including defects X1 to X4 are illustrated in FIGS. 1(a) to (d) respectively. Here, the classification categories of defects X1 to X4 are respectively set to A defect, B defect, C defect, and D defect. On the other hand, images P1' to P4' without defects are illustrated in FIGS. 1(e) to (h) respectively for reference.

FIG. 2 is a schematic diagram showing the overall structure including an example of embodiment of the present invention. The overall structure of the inspection system 100 including the automatic defect classification device 1 of the present invention is shown in FIG. 2 .

The inspection system 100 captures an inspection image P for inspecting the inspection object W, inspects the presence, number, location, defect type, etc. of the defects X in the inspection image P, and outputs the inspection results. Specifically, the inspection system 100 is configured to include, in addition to the automatic defect classification device 1 , an imaging unit 110 , a transport unit 120 , a defect detection unit 130 , a control unit 140 and the like.

The transport unit 110 moves at a predetermined speed or remains stationary at a predetermined place while holding the inspection object W. Specifically, the transport unit 110 includes a workpiece holding unit that holds the inspection object W, an XYθ stage unit that moves and rotates the inspection object W in the horizontal direction, and the like.

The imaging unit 120 captures a moving or stationary inspection object W and outputs an inspection image P. Specifically, the imaging unit 120 includes an optical system body unit 121 including a lens barrel and a lens, an illumination unit 122, and an imaging camera 123. More specifically, the imaging unit 120 may use a black-and-white camera or the like using a low-magnification lens to capture a wide field of view, or a color camera or the like using a high-magnification lens to capture a narrow field of view.

The defect detection unit 130 acquires the inspection image P output from the imaging unit 120 and detects the defect X in the inspection image P. Specifically, the defect detection unit 130 is configured to check whether the defect X is included in the inspection image P, and if the defect X is included, add defect information such as the number, position, and area of the defect , check the result) and output. Furthermore, the defect detection unit 130 may also output a learning defect image Pt which will be described in detail below. More specifically, the defect detection unit 130 may be one that inspects based on a black-and-white image captured with a low-magnification lens, or one that performs an inspection based on a color image captured with a high-magnification lens.

The control unit 140 controls the inspection system 100 as a whole. Specifically, the control unit 140 is connected to the automatic defect classification device 1 , the transport unit 110 , the imaging unit 120 , the defect detection unit 130 and the like, and inputs and outputs control signals to each unit to cause the automatic defect classification device 1 to acquire the required defects. ImagePx.

FIG. 3 is a schematic diagram showing the main part of an example of embodiment of the present invention. Figure 3 shows a schematic diagram of the automatic defect classification device 1 of the present invention.

The automatic defect classification device 1 classifies the types of defects X present in the inspection object W. Specifically, the automatic defect classification device 1 uses the teaching data T to generate a classifier for the defect image, and uses the classifier to automatically assign a classification class to the defect image Px whose classification class is unknown. The teaching material T is assigned by the user in advance to each of the plurality of learning defect images Pt (so-called learning images) acquired in advance, with a classification category clearly indicating which defect classification the defect image Pt belongs to. More specifically, the automatic defect classification device 1 includes a defect image input unit GI, a teaching material input unit 2, an operation input unit 3, a memory unit 4, a processing unit 5, a display unit 6, a result output unit RO, etc., and is controlled by a computer ( Hardware) and its execution program (software) are configured to determine the type of defect X present in the defect image Px detected by the defect detection unit 130 and assign a classification category.

The defect image input unit GI inputs the defect image Px. Specifically, the defect image input unit GI acquires the defect image Px output from the defect detection unit 130 or the like. Furthermore, the defect image input unit GI may acquire an additional learning defect image Pt output from the defect detection unit 130 or the like.

The teaching data input department 2 is the one who inputs the teaching data T. Specifically, the teaching data input unit 2 acquires teaching data T output from an external device, a host computer, or the like. More specifically, the teaching material input unit 2 acquires the teaching material T via a recording medium, a communication line, or the like. Furthermore, a plurality of defect images Pt and the classification categories of the defects X included in the defect images Pt are stored in the teaching data T in association with each other.

The operation input unit 3 accepts the user's cursor (also called pointer) operation, numerical input, and the like. Specifically, the operation input unit 3 is used to select an image displayed on the display unit 6, select or change a setting item, or input a numerical value. Furthermore, the defect image input unit GI, the teaching material input unit 2, and the operation input unit 3 are composed of a part of the input unit DI of the computer.

The memory unit 4 memorizes the teaching data T, defect images Pt, Px, programs used in each process, etc. Specifically, the memory unit 4 is composed of a computer memory, an auxiliary memory device (SSD (Solid State Drive, solid state drive), HDD (Hard Disk Drive, hard disk drive), etc.).

The processing unit 5 performs predetermined processing on a plurality of defect images Pt and unknown defect images Px. Furthermore, the processing unit 5 changes the display format of the defective part Q of the defective image Pt on the display unit 6 and displays it. Specifically, the processing unit 5 is configured to change the display unit based on the above-mentioned classification category to which the defective image assigned by the evaluation classifier Bt belongs based on a pre-registered predetermined program, a user's operation (an example will be described below), etc. The defective image Pt in 6 is displayed in a display format of the defective part Q. More specifically, the processing unit 5 includes a classifier generation unit 51 for teaching material evaluation, a feature amount calculation unit 52, a classification accuracy calculation unit 53, a classification category determination unit 54, a defective part display changing unit 55, and an automatic defect classifier generation unit. The department 57, the defect classification department 59, etc. are composed of a computer (hardware) and its execution program (software).

The teaching material evaluation classifier generating unit 51 generates the evaluation classifier Bt in a stage before generating the automatic defect classifier Ba for assigning a classification category to the defect image Px of which the classification category is unknown. The evaluation classifier Bt is used to evaluate the classification categories assigned to the plurality of defect images Pt included in the teaching material T. Specifically, the teaching material evaluation classifier generation unit 51 performs machine learning based on the teaching material T and generates an evaluation classifier Bt including a decision tree and the like. More specifically, the teaching material evaluation classifier generation unit 51 generates an evaluation classifier Bt including a model (so-called random forest) that generates a plurality of decision trees and determines the decision tree by majority decision or the like. Wait for the predicted results.

The feature amount calculation unit 52 is a unit that calculates feature amounts for each of the plurality of defect images Pt included in the teaching material T based on the evaluation classifier Bt. Specifically, the feature amount calculation unit 52 calculates the frequency of appearance of the feature amount of each classification category based on the prediction result of the decision tree in the evaluation classifier Bt generated by the teaching material evaluation classifier generation unit 51, and calculates the appearance frequency. Standardization, etc., and calculate the weight of each feature quantity in each classification category. Furthermore, the plurality of defect images Pt included in the teaching material T are all grouped according to classification categories, and based on the feature vectors of the image groups in each classified classification category, the values used to determine each classification category are calculated and determined. The allowable standard (for example, the range of normal values) for the degree of consistency of feature quantities (that is, classification accuracy).

The classification accuracy calculation unit 53 calculates, for each of the plurality of defect images Pt included in the teaching material T, whether the characteristic amount of the defect image Pt is the same as or different from the classification category assigned to the defect image Pt. The degree of consistency of each allowable standard of the feature quantity set by the classification category (ie, classification accuracy). Specifically, the classification accuracy calculation unit 53 performs statistical processing, etc., to calculate each defect image for each same classification category classified by the user based on the feature amount of each defect image Pt calculated by the feature amount calculation unit 52 Whether each feature component of the feature vector of Pt falls within the range of normal values is used as the classification accuracy.

The classification category determination unit 54 determines the classification category to which the defect image Pt assigned by the evaluation classifier Bt belongs. Specifically, the classification category determination unit 54 determines which classification category the defect image Pt belongs to for each defect image Pt based on the classification accuracy calculated by the classification accuracy calculation unit 53 .

The defective part display changing unit 55 changes the display mode of the defective part Q of the defective image Pt. Specifically, the defective part display changing unit 55 performs the following processing: performs image processing on the defective image Pt based on preset parameters and programs, and changes the display form of the defective part Q according to each classification category to which the defective image Pt belongs. . For example, if the classification category of the defect image Pt is A defect, the defective part Q is changed to "green", if it is a B defect, it is changed to "yellow", and if it is a C defect, it is changed to "red". In addition, which defect is changed to which color to display is set in advance by the user and can be changed appropriately in advance.

The automatic defect classifier generating unit 57 generates an automatic defect classifier Ba for assigning a classification class to the defect image Px. Specifically, the automatic defect classifier generating unit 57 has the same configuration as the teaching material evaluation classifier generating unit 51, performs machine learning based on the improved teaching material T′, and generates an automatic defect classifier Ba including a decision tree and the like.

The defect category classification unit 59 assigns a classification category to the defect image Px whose classification category is unknown. Specifically, the defect category classification unit 59 assigns a classification category to the defect image Px whose classification category is unknown based on the automatic defect classifier Ba.

The display unit 6 displays a plurality of defective images Pt, classification categories assigned to the defective images Pt, and other information related to the defective images. Specifically, the display unit 6 is composed of an image monitor, a touch panel, or the like (that is, a part of the output unit DO of the computer), and displays the defect image Pt and the like that have been subjected to predetermined processing by the processing unit 5 . More specifically, the display unit 6 is combined with the operation input unit 3 to form the user interface U.

FIG. 4 is an image diagram showing an example of a display unit in an embodiment embodying the present invention. FIG. 4 illustrates the image map GA displayed on the display unit 6 .

The display unit 6 includes a defect image display area G1, an image display setting area G2, a display item setting area G3, a display order setting area G4, a classification information display area G5, etc., and can be operated by the user to change display items, parameters, etc. , confirmation, etc.

The defect image display area G1 is an area for displaying the selected defect image Pt (image number 001 in this example). Specifically, the structure is as follows: at the right end of the defect image display area G1, a vertical scroll bar L11 for vertically scrolling the displayed defect image Pt and up and down movement buttons B11 and B12 are provided, and at the lower end there are provided The horizontal scroll bar L12 and the left and right movement buttons B13 and B14 for horizontally scrolling the defect image Pt make it possible to confirm the portion that cannot be completely displayed (that is, hidden) in the defect image display area G1.

The image display setting area G2 is an area for changing and setting the display magnification and brightness of the defect image Pt displayed in the defect image display area G1. Specifically, the currently set display magnification (×1 in this example) is displayed in the image display setting area G2. Then, when button D21 is pressed, other display magnifications are displayed in the form of a pull-down menu. Then, move or click the cursor C in the drop-down menu to select the magnification to be displayed (for example, ×2, ×5, the entire image, etc.), thereby displaying the changed display magnification. In addition, a brightness setting value display area G21 and a toggle switch L21 are arranged in the image display setting area G2. The current brightness setting value is displayed in the brightness setting value display area G21. The toggle switch L21 is used to change the brightness setting value. By aligning the cursor C with the toggle switch L21 and performing a drag operation to move it left and right, the current brightness setting value is changed, thereby changing the brightness of the defect image Pt displayed in the defect image display area G1.

The display item setting area G3 is an area for setting whether to change or not change the display mode of the defective part Q of the defective image Pt displayed in the defective image display area G1. Specifically, frame button B3 is arranged in display item setting area G3. Moreover, every time the frame button B3 is clicked, the inside of the frame button B3 is alternately switched to white and black, and the display mode of the defective part Q of the defect image Pt is changed in accordance with this switching. More specifically, when the frame button B3 is white, the defective part Q is displayed as it is in the defective image Pt. On the other hand, when the frame button B3 is black, the present invention described in detail below is used to change the display form of the defective part Q of the defective image P so that it is displayed with color and pattern.

The display order setting area G4 is an area for setting the order of the defect images Pt displayed in the defect image display area G1. Specifically, in the display order setting area G4, from the plurality of pre-registered display orders (such as data list ascending order, data list descending order, category ascending order, category descending order, classification accuracy ascending order, classification accuracy descending order,...) Select one to set the display order in which the defect image Pt is displayed. In addition, the so-called ascending order/descending order of the data list refers to a method of displaying the data list in ascending/descending order based on the consecutive numbers assigned to the defect images Pt. In addition, the so-called ascending order/descending order of categories refers to a method of displaying them in ascending/descending order based on the classification category assigned to the defect image Pt. In addition, the classification accuracy ascending order/descending order refers to a method of displaying the classification accuracy in ascending/descending order based on the classification accuracy (degree of agreement) of the classification category assigned to the defect image Pt.

For example, the currently set order (in this case, ascending order of the data list) is displayed in the display standard setting area G4, but when button D41 is pressed, other display orders will be displayed in the form of a drop-down menu. Then, move or click the cursor C in the drop-down menu to select the order to be displayed (for example, ascending order of categories), thereby displaying the changed display order, and the defect image Pt in the defect image display area G1 according to The categories are rearranged in ascending order and displayed. Furthermore, the image number display area G41, 10 rewind buttons B41, 1 rewind button B42, 1 forward button B43, 10 forward buttons 44, etc. are arranged in the display order setting area G4. When any button is clicked, the image number displayed in the image number display area G41 will be changed, and the defect image Pt of the image number will be displayed in the defect image display area G1.

The classification information display area G5 is an area that displays classification information related to the image selected by moving, clicking, etc. the cursor C (hereinafter referred to as the selected image) among the defect images Pt displayed in the defect image display area G1 . Specifically, the classification information display area G5 is provided with an MDC (Multiple Description Coding) display area G51, an MDC candidate display area G52, a classification accuracy display area G53, a classification change target selection area G54, a change button B53, and the like. .

The MDC display area G51 is an area that displays the classification category assigned by the user to the selected image. For example, "NG-A" indicating A defect is displayed in the MDC display area G51.

The MDC candidate display area G52 is an area that displays reclassification candidate categories for the selected image. Specifically, the classification category with the second highest classification accuracy is displayed in the MDC candidate display area G52 as a reclassification candidate category.

The classification accuracy display area G53 displays the classification accuracy (that is, the degree of agreement) of the classification category to which the defect image Pt displayed in the defect image display area G1 belongs. Specifically, in the classification accuracy display area G53, each classification category and classification accuracy are displayed in a table format. Furthermore, a vertical scroll bar L51 for vertically scrolling a plurality of classification categories and up and down movement buttons B51 and B52 are provided at the right end of the classification accuracy display area G53 to confirm that the classification accuracy display area G53 cannot be fully displayed at one time. Other classification categories and classification accuracy within (i.e., hidden).

The classification category change target selection area G54 is an area used to select which classification category to change when changing the classification category assigned to the selected image. For example, the currently assigned classification category (NG-A in this example) is displayed, but when button D51 is pressed, other classification categories (NG-B, C, D,...) will be displayed in the form of a drop-down menu Come out. Then, by moving or clicking the cursor C in the drop-down menu, the classification category to be changed (for example, NG-C) is selected, thereby displaying the classification category of the change target (however, at this point in time, the teaching Data T has not been changed).

The change button B53 is a switch button used to determine the change of the classification category. By pressing the change button B53, the teaching material T is changed by using the classification category displayed in the classification category change target selection area G54 as the classification category of the selected image.

Furthermore, buttons B4 and B5 are arranged on the lower right side of the image GA. When button B4 is pressed, the parameter changes are saved and the displayed window is closed, the teaching material T is improved, and the improved teaching material T' is generated. On the other hand, when button B5 is pressed, the displayed window is closed without saving the parameter changes (that is, the teaching material T is not improved and the teaching material T' is not generated).

The result output unit RO outputs the classification results. Specifically, the classification category (that is, the classification result) automatically assigned by the automatic defect classifier Ba to the defect image Px of unknown classification category acquired by the defect image input unit GI is output to an external device, a host computer, or the like. More specifically, the result output unit RO is composed of a part of the output unit DO of the computer.

[Processing flow] FIG. 5 is a flowchart showing an example of embodiment of the present invention. Figure 5 shows the process of using the automatic defect classification device 1 of the present invention to correct the teaching materials T containing errors in the classification categories assigned by the user, so as to improve the accuracy of automatic defect classification.

First, the teaching data T output from an external device, a host computer, etc. is input from the teaching data input unit 2 (step s1). Specifically, based on the user's instructions, the teaching data T is output from an external device, a host computer, etc.

Next, the teaching material evaluation classifier generating unit 51 generates an evaluation classifier Bt based on the teaching material T (step s2). Specifically, the user presses the "execute" button etc. displayed on any screen etc., thereby starting machine learning based on the current teaching data T to generate the evaluation classifier Bt.

Next, the feature amount calculation unit 52 performs a feature amount calculation process for each of the plurality of defect images Pt included in the teaching material T based on the evaluation classifier Bt (step s3).

Next, the classification accuracy calculation unit 53 calculates, for each of the plurality of defect images Pt included in the teaching material T, the feature amount of the defect image Pt, which is the same as the classification category assigned to the defect image Pt. And the degree of consistency (ie, classification accuracy) of each allowable standard of the feature quantities set for different classification categories (step s4).

Next, the classification category determination unit 54 determines, for each of the plurality of defect images Pt, the classification category to which the defect image Pt assigned by the evaluation classifier Bt belongs (step s5).

Next, the display format of the defective part Q of the defective image Pt is changed and displayed on the display unit 6 (step s6).

Then, the user confirms whether the classification category of the defect image Pt is appropriate and determines whether the classification category included in the teaching material T needs to be changed (ie, corrected) (step s7).

If necessary, change the classification category of one or more defect images Pt (step s8), improve the teaching material T, and generate the improved teaching material T'.

Then, it is determined whether to use the teaching material evaluation classifier generation unit 51 to perform machine learning (relearning) based on the improved teaching material T' and regenerate the evaluation classifier Bt (step s9). If it is necessary to re-learn, repeat the above steps s2 to s9. If it is not necessary to re-learn, then the series of processes will be ended.

Then, the automatic defect classifier generating unit 57 is used to perform machine learning based on the latest state (ie, improved) teaching material T′ to generate an automatic defect classifier Ba. Then, in the automatic defect classification device 1 of the present invention, the defect class classification unit 59 uses the automatic defect classifier Ba to assign a classification class to the defect image Px whose classification class is unknown.

The automatic defect classification device 1 of the present invention has the above-mentioned structure. Therefore, the processing unit 5 can be used to perform predetermined processing on a plurality of defect images Pt, and the display of the defective part Q of the defect image Pt in the display unit 6 can be changed. form to display. Therefore, even if there are many defect images Pt constituting the teaching material T, errors in the classification categories assigned by the user can be easily corrected, and the accuracy of automatic defect classification can be improved.

[About display mode] In terms of embodying the present invention, it is not limited to the above-described configuration. The processing unit 5 may be configured to have a single image display mode and an image list display mode, so that any image selected by the user in the image list display mode can be used. The defect image Pt is displayed in the single image display mode.

The "single image display mode" is a mode in which an image selected from a plurality of defect images Pt classified into the same category is displayed on the display unit 6 . In addition, the image diagram GA displayed on the display unit 6 illustrated in FIG. 4 is equivalent to the display in the "single image display mode", and therefore a detailed description is omitted. On the other hand, the "image list display mode" is a mode in which a plurality of defect images Pt classified into the same category are displayed on the display unit 6 in a matrix arrangement.

FIG. 6 is an image diagram showing an example of a display unit in an embodiment embodying the present invention. FIG. 6 illustrates the image image GB displayed on the display unit 6 in the "image list display mode".

When switching the display mode, buttons B1 and B2 are arranged above the defect image display area G1 of the display unit 6 in advance. Button B1 is used to switch and display the defect image display area G1 to the "single image display mode". On the other hand, button B2 is used to switch and display the defect image display area G1 to the "image list display mode". Specifically, when button B2 is pressed while the image graph GA is in the "single image display mode" as shown in Figure 4, it switches to the "image list display mode" as shown in Figure 6 Image GB, if button B1 is pressed, switches to image GA in the "single image display mode" as shown in Figure 4.

In the "image list display mode", the defect image display area G1 is an area that displays a plurality of defect images Pt and classification categories assigned to the defect images Pt. Specifically, in the defect image display area G1, a plurality of defect images Pt are displayed in a matrix form (also called a brick shape), and a classification category is displayed under the defect images Pt. Furthermore, the right end of the defect image display area G1 is provided with a vertical scroll bar L11 for vertically scrolling the plurality of defect images Pt, and the vertical movement buttons B11 and B12, and the lower end is provided with a vertical scroll bar L11 for vertically scrolling the plurality of defect images Pt. By scrolling the horizontal scroll bar L12 and the left and right movement buttons B13 and B14, it is possible to confirm other defect images Pt that cannot be completely displayed in the defect image display area G1 (that is, are hidden).

"Image list display mode" is displayed based on the display order set in the display order setting area G4 (for example, data list ascending order, data list descending order, category ascending order, category descending order, classification accuracy ascending order, classification accuracy descending order,...) A plurality of defect images Pt. Specifically, in the ascending/descending order of the data list, based on the consecutive numbers assigned to the defect images Pt, a plurality of defect images Pt are arranged and displayed in ascending/descending order from the upper left end to the right and lower sides. On the other hand, in the case of ascending/descending category order, based on the classification category assigned to the defect image Pt, a plurality of defect images Pt are arranged and displayed in ascending/descending order from the upper left end to the right and lower sides. On the other hand, in the ascending/descending order of classification accuracy, based on the classification accuracy (degree of consistency) of the classification category assigned to the defect image Pt, the plurality of defect images Pt are sorted in ascending/descending order from the upper left to the Arrange and display on the right and lower sides. In addition, the "image list display mode" displays the image number of the defective image Pt selected by the user (for example, the user moves the cursor C to the defective image Pt and clicks it) in the image number display area. G41.

[About switching operations] When there are "single image display mode" and "image list display mode", if the user performs the following operations in the "image list display mode", the processing unit 5 switches any defect image Pt selected by the user Displayed for "single image display mode".

Specifically, in the state of the image diagram GB in the "image list display mode" shown in FIG. 6, the user moves the cursor C to the defect image Pt to be displayed enlarged (in this example, from the top 2) from the left and double-click to display the defect image Pt in the "single image display mode". Alternatively, the user moves the cursor C to the defect image Pt to be displayed enlarged (in this example, the second one from the left of the uppermost segment) and clicks it to enter the image selection state and then presses the button B1. This causes the defect image Pt to be displayed in the "single image display mode".

FIG. 7 is an image diagram showing an example of a display unit in an embodiment embodying the present invention. FIG. 7 illustrates an image diagram GC in which any defect image Pt selected by the user (image number 002 in this example) is displayed in the defect image display area G1 of the display unit 6 in the "single image display mode". .

With this configuration, the color, shape, etc. of the defective parts Q of the plurality of defective images Pt can be observed from above in the image list display mode, or the defective image Pt that requires attention can be displayed in the single image display mode. It is better to judge whether the classification category is appropriate. For example, in the image list display mode, comparing the defective image Pt with the same display format (color, shape, etc.) of the defective part Q helps the user to judge whether it is appropriate to be assigned the same classification category. On the other hand, in the single-screen display mode, it is possible to confirm whether the defective part Q displayed in the same display format (color, pattern, etc.) is unnatural (the defective part Q exists in the corresponding phase) while transmitting the defective images Pt one by one. separated place, or the defective part Q does not exist where it should exist, etc.).

Therefore, even if there are many defect images Pt constituting the teaching material T, errors in the classification categories assigned in advance by the user can be easily corrected, and the accuracy of automatic defect classification can be improved.

Furthermore, in the aspect of embodying the present invention, the switching of the display mode as described above is not an essential function, and the same display method as the single screen display mode or the image list display mode can be used for display. Defect image Pt. In this case, it is also easier to correct the error in the classification category than in the case where the display form of the defective part Q is not changed (previous display), thereby improving the accuracy of automatic defect classification.

Furthermore, in the above, the processing unit 5 has been shown to be configured to change the display format of the defective part Q of the defective image in the display unit 6 according to the classification category to which the defective image Pt assigned by the classifier belongs. And display.

However, the processing unit 5 may be configured as follows instead of this structure, or may be configured in addition to this structure to assign the display format of the defective part Q of the defective image in the display unit 6 based on the classifier. The classification accuracy of the classification category to which the defect image Pt belongs is changed and then displayed.

FIG. 8 is an image diagram showing an example of a display unit in an embodiment embodying the present invention. 8(a) to (c) respectively show an example of the defective part Q of the defective image Pt displayed in the defective image display area G1 of the display unit 6. The defect images Pt shown in Figures 8(a) to (c) all belong to the same classification category (for example, classified as A defect), but the classification accuracy is different.

Furthermore, in this example, the hatching (that is, the shape) of the defective part Q is changed and displayed according to the classification accuracy of the classification category, but the color of the defective part Q may also be changed. For example, when the classification accuracy is high, the display format is changed to "green", when the classification accuracy is low, the display format is changed to "red", and when the classification accuracy is medium, the display format is changed to "red". In this case, the display format will be changed to "yellow" and displayed.

With this configuration, the user can recognize the defective images Pt one by one while confirming them in the single screen display mode or confirming a plurality of defective images Pt at once in the image list display mode. The classification accuracy is low, and it can be judged whether to correct the classification category, so it is better.

Therefore, even if there are many defect images Pt constituting the teaching material T, errors in the classification categories assigned in advance by the user can be easily corrected, and the accuracy of automatic defect classification can be improved.

1: Automatic defect classification device 2:Teaching material input department 3: Operation input part 4:Memory Department 5:Processing Department 6:Display part 51: Classifier generation part for teaching material evaluation 52: Feature quantity calculation part 53: Classification accuracy calculation part 54: Classification Category Judgment Department 55: Defect part display change part 57: Automatic defect classifier generation part 59:Defect Category Classification Department 100: Check system 110:Camera Department 120:Transportation Department 121: Optical system body part 122:Lighting Department 123:Camera 130: Defect detection department 140:Control Department B1:Button B2:Button B3:Button B4:Button B5:Button B11,B12: move up and down buttons B13, B14: left and right movement buttons B41: Reverse button B42: Reverse button B43: forward button B44: forward button B51, B52: move up and down buttons B53:Change button Ba: Automatic defect classifier Bt: Classifier for evaluation C:cursor D21:Button D41:Button D51:Button DI: input department DO: output department G1: Defect image display area G2: Image display setting area G3: Display project setting area G4: Display order setting area G5: Classified information display area G21: Brightness setting value display area G41: Image number display area G51: MDC display area G52: MDC candidate display area G53: Classification accuracy display area G54: Classification change target selection area GA: image graph GB: image map GC: image graph GI: Defect image input unit L11: Longitudinal rolling rod L12: Horizontal rolling stick L21: toggle switch L51: Longitudinal rolling rod P: Check image P1~P4: Defect images P1'~P4': images without defects Pt: Defect images for learning (complete classification assignment) Px: Check image (classification category unknown) Q:Defective parts RO: Result Output Department T: Teaching materials (before improvement) T': Teaching materials (improved) U: User interface W: Check object X: Defect X1~X4: Defects s1~s9: steps

1 (a) to (h) are image diagrams showing an example of a defect image in an embodiment embodying the present invention. FIG. 2 is a schematic diagram showing the overall structure including an example of embodiment of the present invention. FIG. 3 is a schematic diagram showing the main part of an example of embodiment of the present invention. FIG. 4 is an image diagram showing an example of a display unit in an embodiment embodying the present invention. FIG. 5 is a flowchart showing an example of embodiment of the present invention. FIG. 6 is an image diagram showing an example of a display unit in an embodiment embodying the present invention. FIG. 7 is an image diagram showing an example of a display unit in an embodiment embodying the present invention. 8(a) to (c) are image diagrams showing another example of the embodiment embodying the present invention.

P1~P4: Defect images

P1'~P4': images without defects

X1~X4: Defects

Claims (3)

An automatic defect classification device that uses teaching materials in advance that the user assigns to each of a plurality of defect images a classification category that clearly indicates which defect classification the defect image belongs to, generates a classifier for the defect image, and uses the The classifier automatically assigns classification categories to defect images with unknown classification categories; the automatic defect classification device is characterized by: The teaching data input department is responsible for inputting the above-mentioned teaching data; A processing unit that performs prescribed processing on the plurality of defective images mentioned above; and A display section that displays the above defective image; and The processing unit changes and displays the display format of the defective part of the defective image on the display unit according to the classification category to which the defective image belongs given by the classifier. An automatic defect classification device that uses teaching materials in advance that the user assigns to each of a plurality of defect images a classification category that clearly indicates which defect classification the defect image belongs to, generates a classifier for the defect image, and uses the The classifier automatically assigns classification categories to defect images with unknown classification categories; the automatic defect classification device is characterized by: The teaching data input department is responsible for inputting the above-mentioned teaching data; A processing unit that performs prescribed processing on the plurality of defective images mentioned above; and A display section that displays the above defective image; and The processing unit changes and displays the defective part of the defective image in the display unit according to the classification accuracy of the classification category to which the defective image belongs given by the classifier. For example, the automatic defect classification device of claim 1 or 2, wherein the above-mentioned processing unit is equipped with: A single image display mode, which causes a selected image from a plurality of the above-mentioned defect images classified into the same category to be displayed on the above-mentioned display unit; and an image list display mode in which a plurality of the defect images classified into the same classification category are arranged in a matrix and displayed on the display unit; and Any defect image selected by the user in the image list display mode is displayed in the single image display mode.