JP2006292615A - Visual examination apparatus, visual inspection method, program for making computer function as visual inspection apparatus, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a visual inspection apparatus for detecting defects, according to the accuracy requested. <P>SOLUTION: Processing, executed by the visual examination apparatus, comprises a step (S402) for receiving the input of requested precision and images; steps (S404, S406) for extracting defect candidates and feature amount; a step (S408) for displaying the amount of feature; a step (S410) for receiving the input of labeling information; a step (S412) for executing the estimated processings of population distribution; a step (S414) for calculating the degree of overlapping among populations; a step (S416) for reading defect information; a step (S418) for calculating the precision; a step for starting learning, when an inputted image does not satisfy the requested precision (YES in S420); and a step (S422) for calculating learning information when the image does not satisfy the requested precision (NO in S420). <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a technique for inspecting the appearance of an object to be inspected. More specifically, the present invention relates to an appearance inspection device that inspects an appearance based on image data acquired by photographing an object to be inspected, an appearance inspection method, a program for causing a computer to function as an appearance inspection device, and its The present invention relates to a computer-readable recording medium storing a program.

  2. Description of the Related Art Conventionally, some appearance inspection apparatuses automatically detect defects by performing image processing on image data obtained by imaging the appearance of an inspection target using an algorithm generated by an optimization method. For example, an apparatus for inspecting the appearance of a semiconductor or liquid crystal substrate, for example, uses an image of a defect such as a crack, a hook, foreign matter adhesion, or unevenness generated on the surface of the substrate as a learning image.

  Such learning requires repetitive calculations and requires a large number of multidimensional feature quantities. Therefore, enormous calculation time is required. Therefore, learning data selection and correction methods have been developed.

  For example, Japanese Patent Laying-Open No. 2003-76976 (Patent Document 1) discloses a pattern matching method using a neural network with high determination accuracy. This method calculates the correlation between feature quantities belonging to different dimensions from multiple learning input patterns, and creates a boundary pattern at a position where the characteristic distance determined by the correlation (here, Mahalanobis distance) is constant. And a step of using the boundary pattern as a learning pattern. According to the above method, when learning multidimensional data, the correlation between the data is calculated, boundary data is created at a position where the characteristic distance determined by the correlation is constant, and more accurate boundary data is created. The Thereby, a pattern matching method with high determination accuracy using a neural network is provided.

Japanese Patent Application Laid-Open No. 11-344450 (Patent Document 2) discloses a defect classification apparatus having a learning function that can classify a defect type from a defect image signal. The apparatus includes a defect image detecting unit that images a plurality of teaching defects in advance and detects a plurality of teaching defect images, and a plurality of teachings based on the plurality of teaching defect images detected by the defect image detecting unit. A calculation unit that classifies each of the defects for use and assigns a category; a plurality of defect images for teaching detected by the defect image detection unit; and a category that is classified for the plurality of teaching defects by the calculation unit. A display unit for displaying, a learning unit for acquiring teaching data indicating a correspondence relationship between a plurality of teaching defects and a category corresponding to the type of the defect, and storing the data in the storage unit; and inspecting the inspection object Based on the defect image detection unit that detects the defect image detected by the defect image detection unit by imaging the detected defect and detects the defect image of the inspection target, the storage unit of the learning unit From the stored teaching data And a classification unit that includes a calculation unit that classifies the type of defect.
JP 2003-76976 A JP 11-344450 A

  All of the techniques disclosed in the above-mentioned patent documents create learning data or present it to an operator for an accurate pattern matching method and inspection apparatus.

  Here, the method disclosed in Patent Document 1 creates learning data based on the distance obtained from the correlation between feature quantities of different dimensions. Specifically, the learning data is a collection of a set having a constant Mahalanobis distance from the average of the feature quantities of the learning data. Therefore, although the feature amount level of the learning data can be determined, there is a problem that it is difficult to determine the number thereof.

  Further, the apparatus disclosed in Patent Document 2 uses a classification parameter represented by a discriminant function between categories in a feature amount space for a learning defect feature amount given a category by an operator. By diagnosing significance and presenting the result to the operator, the operator corrects or excludes the learning data. Therefore, there is a problem that it is difficult to determine the pattern matching method with high accuracy in consideration of the statistical distribution of defects and the feature amount level and number of learning data necessary for the inspection apparatus.

  The present invention has been made to solve the above-mentioned problems, and its purpose is to statistically evaluate the feature quantity of true / pseudo defects in an image and to use an algorithm generated by an optimization method. In the pattern recognition method and the inspection apparatus using these, the approximate learning evaluation accuracy is estimated in advance before learning, whether or not the accuracy required by the user is reached is determined before learning, and according to the result It is to provide a highly reliable appearance inspection apparatus according to the required accuracy by enabling addition of learning data.

  Another object of the present invention is to estimate the approximate learning evaluation accuracy in advance before learning, determine whether the accuracy required by the user is reached before learning, and add learning data according to the result Thus, it is to provide a program for causing a computer to function as a highly reliable appearance inspection apparatus according to the required accuracy.

  In order to solve the above-described problems, according to one aspect of the present invention, the appearance inspection apparatus includes input means for receiving data input from the outside. The data includes image data acquired by imaging the inspection object, and required accuracy required for evaluating a feature amount expressed based on the image data as a feature of the image of the inspection object. The appearance inspection apparatus performs predetermined image processing on the input image data, thereby extracting defect candidate data for specifying defect candidates in the inspection object, and defect candidate data The feature amount calculating means for calculating the feature amount of the defect candidate and the display means for displaying the feature amount are provided. The input means further receives input of data that has been labeled to classify defect candidates according to the feature amount. The labeled data includes data for classifying defect candidates as either true or pseudo defects, and pseudo defects are classified as non-true defect candidates as defects. It is what is done. The appearance inspection apparatus is based on the calculation means for calculating the degree of overlap of defect candidate distribution based on the labeling input, the number of pseudo defects, the number of true defects, and the degree of overlap of distribution. An accuracy calculating means for calculating the accuracy for evaluating the distribution of the feature quantity, a determining means for determining whether or not to perform image data learning based on the required accuracy and the calculated accuracy, and a determining means Output means for outputting the result of the determination.

  Preferably, the calculation means includes an estimation means for estimating the population distribution from the labeled defects based on the labeling input, and a multiplicity calculation means for calculating the degree of overlap based on the population distribution. .

  Preferably, the determining means is a comparing means for comparing the calculated accuracy with the required accuracy, and a selecting means for selecting either the start of learning or the calculation of information necessary for learning based on the comparison result. Including.

  Preferably, the determination unit determines to start learning when the calculated accuracy satisfies the required accuracy. The appearance inspection apparatus further includes a learning information calculation unit that calculates information necessary for learning based on the required accuracy and the calculated accuracy when the determination unit determines not to start learning. The output means includes information output means for outputting information necessary for the calculated learning.

  Preferably, the learning information calculation unit includes a generation unit that calculates image data for displaying notification information for indicating a range of a feature amount necessary to satisfy the required accuracy. The information output means includes display means for displaying notification information based on the image data.

  Preferably, the appearance inspection apparatus further includes defect extraction means for extracting a true defect and a pseudo defect from a defect candidate based on a predetermined criterion. The generation unit generates display data for displaying the number of true defects and the number of pseudo defects extracted by the defect extraction unit based on the result of the extraction process by the defect extraction unit. The display means displays the number of true defects and the number of pseudo defects extracted by the defect extraction means based on the display data.

  Preferably, the generation unit has a relationship between a predetermined first evaluation criterion and a predetermined second evaluation criterion that is different from the first evaluation criterion based on the labeled data. Data calculating means for calculating data for drawing a curve representing the receiver operating characteristic to be shown; range calculating means for calculating a range below a predetermined accuracy based on data for drawing the curve; and feature quantity Based on the reflection means for reflecting the calculated range in the space, the range reflected in the feature space, and the defect information identified based on the defects extracted from the image data. Image data generating means for generating range data for indicating the range of the feature amount. The display means displays the required feature amount range based on the range data.

  Preferably, the defect information includes the number of defects.

  Preferably, the range calculation means specifies a plurality of line segments for constituting a curve, specifies a vertex constituted by an adjacent line segment for each of the plurality of line segments, and each of the specified vertices The distance between vertices is calculated, the calculated distances are sorted, and a range below a predetermined range is calculated based on the sorted distances.

  Preferably, the accuracy calculation means calculates the accuracy for evaluating the distribution of the feature amount either during learning or after the end of learning.

  According to another aspect of the present invention, the appearance inspection method includes an input step of receiving data input from the outside. The data includes image data acquired by imaging the inspection object, and required accuracy required for evaluating a feature amount expressed based on the image data as a feature of the image of the inspection object. In the appearance inspection method, an extraction step for extracting defect candidate data for specifying a defect candidate in an inspection object by executing predetermined image processing on input image data, and defect candidate data The feature amount calculating step for calculating the feature amount of the defect candidate based on the above, the display step for displaying the feature amount, and the input of the labeled data for classifying the defect candidate according to the feature amount Receiving. The labeled data includes data for classifying defect candidates as either true defects or pseudo defects. A pseudo defect is a defect candidate that is not a true defect is classified as a defect. The visual inspection method further includes a calculation step for calculating the degree of overlap of defect candidate distributions based on labeling input, the number of pseudo defects, the number of true defects, and the degree of overlap of distributions. An accuracy calculation step for calculating accuracy for evaluating the distribution of the feature amount, a determination step for determining whether or not to perform image data learning based on the required accuracy and the calculated accuracy, and a determination An output step for outputting a result of the determination by the step.

  Preferably, the calculating step includes a step of estimating a population distribution from the labeled defects based on a labeling input, and a step of calculating an overlapping degree based on the population distribution.

  Preferably, the determining step includes a step of comparing the calculated accuracy with the required accuracy, and a step of selecting one of the start of learning and the calculation of information necessary for learning based on the comparison result. Including.

  Preferably, the determination step determines the start of learning when the calculated accuracy satisfies the required accuracy. The appearance inspection method further includes a step of calculating information necessary for learning based on the required accuracy and the calculated accuracy when the determination step does not determine the start of learning. The output step outputs information necessary for the calculated learning.

  Preferably, the accuracy calculation step calculates the accuracy for evaluating the distribution of the feature amount either during learning or after the end of learning.

  According to another aspect of the present invention, the program causes a computer to function as an appearance inspection device. This program causes a computer to execute a step of receiving data input from the outside. The data includes image data acquired by imaging the inspection object, and required accuracy required for evaluating a feature amount expressed based on the image data as a feature of the image of the inspection object. The program executes a predetermined image processing on the input image data to the computer, thereby extracting defect candidate data for specifying a defect candidate in the inspection object; and defect candidate data And a step of calculating a feature amount, a display step of displaying the feature amount, and a step of receiving input of data that has been labeled to classify defect candidates according to the feature amount. . The labeled data includes data for classifying defect candidates as either true defects or pseudo defects. A pseudo defect is a defect candidate that is not a true defect is classified as a defect. Based on the labeling input, the program calculates the degree of overlap of the defect candidate distribution, the number of pseudo defects, the number of true defects, and the degree of distribution overlap. A step of calculating accuracy for evaluating the distribution of the feature amount, a step of determining whether or not to perform learning of the image data based on the required accuracy and the calculated accuracy, and a result of the determination by the determination step Are executed.

  According to still another aspect of the present invention, a computer-readable recording medium stores the above program.

  According to the appearance inspection apparatus according to the present invention, the learning evaluation accuracy is estimated in advance before learning. The appearance inspection device determines whether or not the accuracy is the accuracy required by the user of the device before learning. Since the appearance inspection apparatus can display the number of learning data satisfying the required accuracy and other information necessary for learning, the user can cause the appearance inspection apparatus to perform relearning as necessary. Thereby, it is possible to provide a highly reliable appearance inspection apparatus according to the required accuracy.

  According to the apparatus for executing the appearance inspection method according to the present invention, the learning evaluation accuracy is estimated in advance before learning. It is determined before learning whether the accuracy is the accuracy required by the user of the method. Since the number of learning data satisfying the required accuracy and other information necessary for learning are displayed, the user can cause the apparatus to perform relearning as necessary. Thereby, it is possible to provide a highly reliable appearance inspection method according to the required accuracy.

  According to the program according to the present invention, the computer estimates the learning evaluation accuracy in advance before learning. The computer determines whether or not the accuracy is the accuracy required by the computer user before learning. Since the computer can display the number of learning data satisfying the required accuracy and other information necessary for learning, the user can cause the computer to perform relearning as necessary. As a result, it is possible to provide a program for causing a computer to function as a highly reliable appearance inspection apparatus according to required accuracy.

  According to the program according to the present invention, it is possible to cause a computer to function as a highly reliable appearance inspection apparatus according to required accuracy.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

  With reference to FIG. 1, an inspection system 10 including an appearance inspection apparatus according to an embodiment of the present invention will be described. FIG. 1 is a block diagram showing the system configuration of the inspection system 10.

  The inspection system 10 includes a stage 101, a light source 103, a CCD (Charge-coupled device) camera 104, a controller 105, and an appearance inspection apparatus 100. On the stage 101, an object 102 that can be an object of appearance inspection is carried into a predetermined position. The object 102 is unloaded from the stage 101 after a predetermined inspection.

  The light source 103 emits light to the stage 101. When the target object 102 is carried into the stage 101, the target object 102 is supplied with a light amount that can be photographed by the CCD camera 104. Note that the amount of light supplied to the object 102 can be changed by the controller 105 described later.

  The CCD camera 104 images the object 102 and outputs an image signal. The image signal is input to the appearance inspection apparatus 100 via a cable. The input data is stored in a predetermined storage area of a storage device provided in the appearance inspection apparatus 100 as will be described later.

  The controller 105 controls the operations of the stage 101, the light source 103, the CCD camera 104, and the appearance inspection apparatus 100. More specifically, the controller 105 controls the positional relationship among the stage 101, the light source 103, and the CCD camera 104 in order to accurately photograph the object 102. This control is performed based on whether or not the object 102 carried into the stage 101 is in focus according to an image signal output from the CCD camera 104, for example. Alternatively, when the user inputs an instruction to the appearance inspection apparatus 100, the appearance inspection apparatus 100 outputs a command for controlling the positional relationship to the controller 105 according to the instruction.

  Referring again to FIG. 1, the appearance inspection apparatus 100 includes an input unit 111, a storage device 107, an image processing unit 108, a calculation unit 110, a main control unit 109, and a display 106. The input unit 111 receives data input from the outside, and sends the data to the main control unit 109 or the like via a signal line (not shown). The storage device 107 stores data in a nonvolatile and volatile manner. The image processing unit 108 executes target image processing in accordance with preset parameters. The calculation unit 110 executes a predetermined calculation in the appearance inspection apparatus 100. The main control unit 109 controls the internal operation of the appearance inspection apparatus 100 based on a control signal input from the controller 105. The display 106 displays a captured image of the object 102 and other image processing results based on data written to a display memory (not shown) by the main control unit 109.

  The image processing unit 108, the main control unit 109, and the calculation unit 110 are realized in hardware by a system circuit configured to execute a predetermined process, for example. Alternatively, as will be described later, it is realized by causing a CPU (Central Processing Unit) to execute a program for realizing each process described later.

  With reference to FIG. 2, the data structure of appearance inspection apparatus 100 according to the present embodiment will be described. FIG. 2 is a diagram illustrating an aspect of data storage in storage device 107 of appearance inspection apparatus 100.

  The storage device 107 includes a table 200 that stores accuracy calculated by processing to be described later, and a program for realizing the appearance inspection device 100. The table 200 includes each corresponding accuracy using, for example, the degree of overlap and the number of defects divided by a predetermined interval as keys. The calculation of this accuracy will be described later. The storage device 107 also stores an accuracy calculation program for calculating the accuracy and a determination program for executing a learning determination by the appearance inspection device 100 in the areas 210 and 220, respectively. Note that the data storage mode in the appearance inspection apparatus 100 is not limited to that shown in FIG.

  With reference to FIG. 3, a control structure of appearance inspection apparatus 100 according to the present embodiment will be described. FIG. 3 is a flowchart illustrating a procedure of processing executed by the arithmetic unit 110 included in the appearance inspection apparatus 100. That is, when the arithmetic unit 110 is configured as a circuit for realizing each process, the following process is executed when the circuit starts a predetermined operation. Further, when the calculation unit 110 is realized by a CPU, the processing is realized when a program that realizes each processing is executed by the CPU.

  In step S <b> 310, calculation unit 110 receives input of discrete data input via input unit 111. In step S320, operation unit 110 calculates distribution data representing the distribution of the population of discrete data. In step S330, calculation unit 110 calculates the degree of overlap based on the calculated distribution data.

  In step S340, calculation unit 110 calculates accuracy corresponding to the degree of overlap and the number of defects. In step S350, arithmetic unit 110 registers the calculated accuracy as a reference table 200 in a predetermined area of storage device 107.

  With reference to FIG. 4, the control structure of the appearance inspection apparatus 100 will be further described. FIG. 4 is a flowchart illustrating a procedure of processes executed by the image processing unit 108, the main control unit 109, and the calculation unit 110, respectively. The following processing may also be realized by hardware as described above, or may be realized by software by the operation of the CPU.

  In step S <b> 402, calculation unit 110 receives input of requested accuracy and image data requested by the user via input unit 111. In step S404, operation unit 110 performs predetermined image processing on the input image data, and extracts defect candidates.

  In step S406, operation unit 110 extracts a feature amount from the extracted defect candidate image data. Here, the feature amount refers to data for evaluating a feature represented based on image data acquired by photographing as an image feature of the inspection object 102. The feature value is a measured value used in particle analysis, for example, circumference, area, area ratio, circularity coefficient, equivalent circular diameter, zeroth moment, first moment, second moment, maximum minimum concentration, total concentration, average Including density, standard deviation of density, horizontal / vertical ferret diameter, horizontal / vertical ferret diameter ratio, maximum / minimum ferret diameter, maximum ferret diameter angle, maximum / minimum ferret diameter ratio, and the like.

  In step S <b> 408, the main control unit 109 stores the feature amount in a memory area used for display on the display 106, and the feature amount is displayed on the display 106. In step S410, calculation unit 110 accepts input of labeling information for a defect candidate via input unit 111. In step S412, operation unit 110 executes a process for estimating the distribution of the population for each labeled defect candidate.

  In step S414, operation unit 110 calculates the estimated degree of overlap between the populations. In step S416, operation unit 110 reads defect information stored in a predetermined area of storage device 107. In step S418, calculation unit 110 calculates the accuracy of the input image data using the degree of overlap, the defect information, and the number of defects.

  In step S420, main controller 109 determines whether or not the input image data satisfies the input required accuracy. This determination is made, for example, by comparing the required accuracy input in step S402 with the accuracy calculated in step S418. If main control unit 109 determines that the input image data satisfies the required accuracy (YES in step S420), the process proceeds to step S430. If not (NO in step S420), the process proceeds to step S422.

  In step S422, operation unit 110 calculates learning information necessary to satisfy the input required accuracy. In step S424, the main control unit 109 writes the calculated learning information in a predetermined area of a display memory (not shown) in order to display the learning information in the predetermined area of the display 106. As a result, the display 106 displays learning information. In step S430, main controller 109 starts learning. Thereafter, the process ends.

  Here, with reference to FIGS. 5 to 9, analysis of receiver operating characteristics (hereinafter referred to as ROC (Receiver Operating Characteristics)) used by the appearance inspection apparatus 100 according to the present embodiment will be described. FIG. 5 is a diagram showing the relationship between the detected defect and the probability density at the time of detection.

  Referring to FIG. 5, appearance inspection apparatus 100 detects a defect based on image data input from CCD camera 104. This detection result is detected by being divided into a true defect 510 and a pseudo defect 530. Here, the pseudo defect (hereinafter, “pseudo defect”) refers to image data that is erroneously determined to be a defect even though it is not a defect. In this case, an area identified as a defect is specified by applying the discriminator 550, which is a criterion, to the detection result of the defect. For example, this region includes a region 520 that is recognized as a true defect and a region 540 that is identified as a pseudo defect. The area of the region 520 is calculated as a recognition rate of true defects.

  As shown in FIG. 5, when the value of the discriminator 550 is further increased, the recognition rate is lowered. In this case, since the area of the region 540 is also reduced, the overdetection rate is accordingly reduced. On the other hand, when the discriminator 550 is set close to the region of the pseudo defect 530, the area of the region 520 is increased, and the area of the region 540 is also increased. That is, the overdetection rate also increases.

  FIG. 6 is a diagram showing the relationship between the overdetection rate and the recognition rate as an ROC curve 600 for representing ROC characteristics. In this case, the ideal detection result is recognition rate = 1.0 and overdetection rate = 0. That is, in FIG. 6, the upper left point 610 is an ideal detection result. Therefore, the closer the ROC curve 610 is to the upper left, the better the accuracy of the detection result.

  FIG. 7 is a diagram illustrating the relationship between the overdetection rate and the recognition rate on the plane representing the ROC curve according to the number of samples. When the number of samples is small, as indicated by a broken line 710, the boundary line for specifying the detection result is expressed as a stepped shape having a large degree of change. When the number of samples increases, for example, as indicated by a dotted line 720, the boundary line for specifying the detection result is represented as a stepped shape having a smaller degree of change than the broken line 710. That is, the dotted line 720 is directed in the direction of the point 610 (overdetection rate = 0, recognition rate = 1.0) as a whole than the broken line 710.

  When the number of samples further increases, the shape of the boundary line becomes smoother, as indicated by a one-dot chain line 730. If there are an infinite number of samples, curves representing the recognition rate and the overdetection rate are smoothly drawn as indicated by an arc 740.

  FIG. 8 is a diagram for explaining the degree of distribution overlap used by the appearance inspection apparatus 100. Here, the distribution is represented by probability density, for example, but may be a histogram representing the frequency of detection results.

  When the appearance inspection apparatus 100 executes the defect extraction process based on the input image signal, the extraction results are represented as, for example, a distribution 810 and a distribution 840, respectively. Here, the distribution 810 represents the distribution of pseudo defects. Distribution 840 represents the distribution of true defects. When the respective distributions are calculated, average values 820 and 850 of the respective distributions are calculated based on the respective extraction results.

  Here, assuming that the standard deviation of the distribution of pseudo defects is σ (1), an average region from 820 to 3 × σ (1) is specified as a range where the pseudo defects exist. Similarly, a region having an average 850 to standard deviation σ (2) with respect to the true defect distribution 840 is specified as a range where the true defect distribution exists. In the present embodiment, since data in the case of standard deviation σ (1)> σ (2) is used, the overlapping degree of each distribution is as shown in FIG.

  FIG. 9 is a diagram for explaining a procedure for calculating information necessary for learning displayed on the display 106.

  When the relationship between the overdetection rate and the recognition rate is extracted based on the image data acquired by photographing, the appearance inspection apparatus 100 uses the overdetection rate, the recognition rate, and the position of the discriminator 550 as information necessary for learning. And are specified respectively. For example, the point 910 is expressed as an overdetection rate = F (1), a recognition rate = T (1), and a position of the classifier 550 = R (1).

  Here, when the position of the discriminator 550 is moved to, for example, R (2) in order to change the determination criterion, the area is over-detection rate = F (2), recognition rate = T (2). In this case, T (1) = T (2).

  Further, when the position of the discriminator 550 is moved to R (3), the area is detected as overdetection rate = F (3), recognition rate = T (3), Represented as position = R (3). In this case, since the overdetection rate at the point 930 is the same as the overdetection rate at the point 920, the value of F (2) and the value of F (3) are the same.

  With reference to FIG. 10, the display mode in the appearance inspection apparatus 100 will be described. FIG. 10 is a diagram illustrating an aspect of a screen displayed on display 106.

  The display 106 displays a diagram representing the distribution of defects and a screen representing levels required for true defects and pseudo defects. That is, when the appearance inspection apparatus 100 executes the defect extraction process based on the image signal output from the CCD camera 104, the true defect and the pseudo defect are displayed for each specimen.

  In the example shown in FIG. 10, the true defect is represented as “x”. Pseudo defects are represented as “◯”. In addition, an upper limit value 1000 and a lower limit value 1002 are displayed in the range of the discriminator that affects the overdetection rate and the recognition rate. Therefore, the area 1020 is the range of the discriminator.

  Information that needs to be learned for each defect is displayed in area 1062 as a necessary minimum level, for example. The data displayed in area 1062 is displayed as corresponding to the data of true defect 1060. Similarly, the required maximum level is displayed in area 1012. Data displayed in area 1012 corresponds to true defect 1010. In this way, since data corresponding to each value is displayed, the user of the appearance inspection apparatus 100 can easily determine how much learning is required. Specifically, the user can recognize what kind of learning data is necessary. For example, the level of learning data, for example, the strength and number of defects is displayed. A number that satisfies the required accuracy is referenced in table 200 and read.

  Here, a computer system 1100 that implements the appearance inspection apparatus 100 will be described with reference to FIG. FIG. 11 is a block diagram showing a hardware configuration of computer system 1100.

  The computer system 1100 includes a CPU 1110 connected to each other via a data bus, a mouse 1120 and a keyboard 1130 for receiving an instruction input by a user, and input data or data temporarily generated by execution of a predetermined process. A RAM (Random Access Memory) 1140 to be stored, a hard disk 1150 capable of storing a large amount of data in a nonvolatile manner, a CD-ROM (Compact Disk-Read Only Memory) drive device 1160, a monitor 1180, and a communication IF (Interface) 1190. A CD-ROM 1162 is attached to the CD-ROM drive device 1160. Processing in the computer system 1100 functioning as the appearance inspection apparatus 100 is realized by each hardware and software executed by the CPU 1110. Such software may be stored in the RAM 1140 or the hard disk 1150, or may be stored and distributed in the CD-ROM 1162 or other storage medium, and read from the storage medium by the CD-ROM drive 1160 or other reader. May be temporarily stored in the hard disk 1150. The software is read from the RAM 1140 or the hard disk 1150 and executed by the CPU 1110. The hardware configuration of the computer system 1100 shown in FIG. 11 is a general one. It can be said that the most essential part of the present invention is software stored in the RAM 1140, the hard disk 1150, the CD-ROM 1162, and other storage media. Since the operation of each hardware of computer system 1100 is well known, detailed description will not be repeated.

  With reference to FIG. 12, a functional configuration of a computer system 1100 that realizes the appearance inspection apparatus 100 will be described. FIG. 12 is a diagram illustrating a configuration of functions realized by CPU 1110 by execution of software.

  The CPU 1110 includes an input / output unit 1210, an extraction unit 1220, a feature amount calculation unit 1230, a calculation unit 1240, an accuracy calculation unit 1250, and a determination unit 1260.

  The input / output unit 1210 receives data input via a data bus. The input / output unit 1210 writes data generated in the CPU 1110 to the RAM 1140. The extraction unit 1220 performs predetermined image processing on the input image data, thereby extracting defect candidate data for specifying defect candidates in the inspection object. The feature amount calculator 1230 calculates a feature amount based on the defect candidate data. The calculation unit 1240 calculates the degree of overlap of defect candidate distributions based on the labeling input. The accuracy calculation unit 1250 calculates the accuracy for evaluating the distribution of the feature amount based on the number of pseudo defects, the number of true defects, and the degree of distribution overlap. The determination unit 1260 determines whether or not to perform image data learning based on the required accuracy and the calculated accuracy.

  As described above, according to the appearance inspection apparatus 100 according to the embodiment of the present invention, the learning evaluation accuracy is estimated in advance before learning. The appearance inspection apparatus 100 determines whether or not the accuracy is the accuracy required by the operator before learning. Since the appearance inspection apparatus 100 can display the number of learning data satisfying the required accuracy and other information necessary for learning, the operator can cause the appearance inspection apparatus 100 to perform relearning as necessary. As a result, the appearance inspection apparatus 100 can realize inspection of defects in the appearance of the inspection object with high accuracy.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

1 is a block diagram illustrating a system configuration of an inspection system 10 including an appearance inspection apparatus according to an embodiment of the present invention. It is a figure showing the one aspect | mode of the storage of the data in the memory | storage device 107 of the external appearance inspection apparatus 100 which concerns on embodiment of this invention. It is a flowchart (the 1) showing the procedure of the process which the external appearance inspection apparatus 100 which concerns on embodiment of this invention performs. It is a flowchart (the 2) showing the procedure of the process which the external appearance inspection apparatus 100 which concerns on embodiment of this invention performs. It is FIG. (1) for demonstrating the ROC analysis which the external appearance inspection apparatus 100 which concerns on embodiment of this invention uses. It is FIG. (2) for demonstrating the ROC analysis which the external appearance inspection apparatus 100 which concerns on embodiment of this invention uses. It is FIG. (3) for demonstrating the ROC analysis which the external appearance inspection apparatus 100 which concerns on embodiment of this invention uses. It is FIG. (4) for demonstrating the ROC analysis which the external appearance inspection apparatus 100 which concerns on embodiment of this invention uses. It is FIG. (5) for demonstrating the ROC analysis which the external appearance inspection apparatus 100 which concerns on embodiment of this invention uses. It is a figure showing the one aspect | mode of the screen which the display 106 of the external appearance inspection apparatus 100 which concerns on embodiment of this invention displays. It is a block diagram showing the hardware constitutions of the computer system 1100 which implement | achieves the visual inspection apparatus 100 which concerns on embodiment of this invention. FIG. 11 is a diagram illustrating a functional configuration realized by execution of software by a CPU 1110 of a computer system 1100.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 Inspection system, 100 Appearance inspection apparatus, 101 stage, 102 Object, 103 Light source, 104 CCD camera, 105 Controller, 106 Display, 107 Storage device, 108 Image processing part, 109 Main control part, 110 Calculation part, 111 Input part 200 table, 1100 computer system, 1110 CPU, 1120 mouse, 1130 keyboard, 1140 RAM, 1150 hard disk, 1160 CD-ROM drive, 1162 CD-ROM, 1190 communication IF, 1180 monitor.

Claims (17)

Input means for receiving input of data from the outside, wherein the data evaluates image data acquired by imaging the inspection object and a feature amount represented based on the image data as a characteristic of the image of the inspection object Including the required accuracy required for
Extraction means for extracting defect candidate data for specifying defect candidates in the inspection object by executing predetermined image processing on the input image data;
Feature amount calculating means for calculating the feature amount of the defect candidate based on the defect candidate data;
Display means for displaying the feature amount,
The input means further receives input of data that has been labeled to classify the defect candidates according to the feature quantity, and the labeled data indicates that the defect candidates are true. Including data for classifying as either a defect or a pseudo defect, wherein the pseudo defect is a candidate for a defect that is not a true defect, and is classified as a defect;
Calculation means for calculating the degree of overlap of the defect candidate distribution based on the labeling input;
An accuracy calculating means for calculating an accuracy for evaluating the distribution of the feature amount based on the number of the pseudo defects, the number of the true defects, and the degree of overlap of the distributions;
Determination means for determining whether or not to perform learning of image data based on the required accuracy and the calculated accuracy;
An appearance inspection apparatus comprising: an output unit that outputs a result of determination by the determination unit. The calculating means includes
Estimating means for estimating a population distribution from the labeled defects based on the labeling input;
The visual inspection apparatus according to claim 1, further comprising: a degree-of-overlap calculation unit that calculates the degree of overlap based on the distribution of the population. The determination means includes
A comparing means for comparing the calculated accuracy with the required accuracy;
The appearance inspection apparatus according to claim 1, further comprising a selection unit that selects either the start of the learning or the calculation of information necessary for the learning based on the comparison result. The determination means determines to start the learning when the calculated accuracy satisfies the required accuracy,
The appearance inspection apparatus, when the determination unit determines not to start the learning, learning information calculation unit that calculates information necessary for the learning based on the required accuracy and the calculated accuracy Further comprising
The appearance inspection apparatus according to claim 1, wherein the output unit includes an information output unit that outputs information calculated for the calculated learning. The learning information calculation means includes generation means for calculating image data for displaying notification information for indicating a range of the feature amount necessary to satisfy the required accuracy,
The appearance inspection apparatus according to claim 4, wherein the information output unit includes a display unit that displays the notification information based on the image data. The appearance inspection apparatus further includes defect extraction means for extracting the true defect and the pseudo defect based on a predetermined criterion from the defect candidates,
The generation unit generates display data for displaying the number of true defects and the number of pseudo defects extracted by the defect extraction unit based on the result of the extraction process by the defect extraction unit;
The visual inspection apparatus according to claim 5, wherein the display unit displays the number of true defects and the number of pseudo defects extracted by the defect extraction unit based on the display data. The generating means includes
A receiver operating characteristic indicating a relationship between a predetermined first evaluation criterion and a predetermined second evaluation criterion different from the first evaluation criterion based on the labeled data. Data calculating means for calculating data for drawing a curve to represent;
Based on data for drawing the curve, range calculation means for calculating a range below a predetermined accuracy;
Reflecting means for reflecting the calculated range in the feature amount space;
Based on the range reflected in the feature amount space and the defect information specified based on the defect extracted from the image data, range data for indicating the required feature amount range is generated. Image data generating means,
The visual inspection apparatus according to claim 6, wherein the display unit displays the required feature amount range based on the range data.   The visual inspection apparatus according to claim 7, wherein the defect information includes the number of defects.   The range calculation means specifies a plurality of line segments for configuring the curve, specifies a vertex constituted by an adjacent line segment for each of the plurality of line segments, and each of the specified vertices The visual inspection according to claim 7, wherein a distance between vertices is calculated, the calculated distances are sorted, and a range that is less than the predetermined range is calculated based on the sorted distances. apparatus.   The appearance inspection apparatus according to claim 1, wherein the accuracy calculation unit calculates an accuracy for evaluating the distribution of the feature amount either at the time of learning or after the end of the learning. An input step of receiving data input from the outside, wherein the data evaluates image data acquired by imaging the inspection object and a feature amount represented based on the image data as a characteristic of the image of the inspection object; Including the required accuracy required for
An extraction step for extracting defect candidate data for specifying a defect candidate in the inspection object by executing predetermined image processing on the input image data;
A feature amount calculating step for calculating a feature amount of the defect candidate based on the defect candidate data;
A display step for displaying the feature amount;
Receiving the input of data that has been labeled to classify the defect candidates according to the feature quantity, and the data that has been labeled has the defect candidates as true defects. Including data for classifying as any of the pseudo defects, wherein the pseudo defects are those in which the defect candidates that are not true defects are classified as defects,
A calculation step of calculating an overlapping degree of the defect candidate distribution based on the labeling input;
An accuracy calculating step of calculating an accuracy for evaluating the distribution of the feature amount based on the number of the pseudo defects, the number of the true defects, and the degree of overlap of the distributions;
A determination step of determining whether to perform learning of image data based on the required accuracy and the calculated accuracy;
A visual inspection method comprising: an output step of outputting a result of determination by the determination step. The calculating step includes:
Estimating a population distribution from the labeled defects based on the labeling input;
The visual inspection method according to claim 11, further comprising: calculating the degree of overlap based on the distribution of the population. The determination step includes
Comparing the calculated accuracy with the required accuracy;
The appearance inspection method according to claim 11, comprising: selecting either the start of the learning or the calculation of information necessary for the learning based on the result of the comparison. The determining step determines the start of learning when the calculated accuracy satisfies the required accuracy,
The visual inspection method further includes a step of calculating information necessary for the learning based on the required accuracy and the calculated accuracy when the determination step does not determine the start of the learning,
The visual inspection method according to claim 11, wherein the output step outputs the information necessary for the calculated learning.   The appearance inspection method according to claim 11, wherein the accuracy calculating step calculates an accuracy for evaluating the distribution of the feature quantity either at the time of learning or after the end of the learning. A program for causing a computer to function as an appearance inspection apparatus, wherein the program is
A step of receiving data input from outside is executed, and the data evaluates image data acquired by imaging the inspection object and a feature amount represented based on the image data as a characteristic of the image of the inspection object. Including the required accuracy required for
Extracting defect candidate data for specifying a defect candidate in the inspection object by executing predetermined image processing on the input image data; and
Calculating the feature amount based on the defect candidate data;
A display step for displaying the feature amount;
Receiving the input of data that has been labeled to classify the defect candidates according to the feature amount, and the labeled data is used to identify the defect candidates as true defects. And the data for classifying as a pseudo defect, the pseudo defect is a candidate for a defect that is not a true defect, and is classified as a defect,
Calculating an overlap degree of the distribution of the defect candidates based on the labeling input;
Calculating accuracy for evaluating the distribution of the feature quantity based on the number of pseudo defects, the number of true defects, and the degree of overlap of the distributions;
Determining whether to perform learning of image data based on the required accuracy and the calculated accuracy;
And a step of outputting a result of determination by the determination step.   A computer-readable recording medium storing the program according to claim 16.