JP2006293820A - Appearance inspection device, appearance inspection method, and program for causing computer to function as appearance inspection device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an appearance inspection device capable of outputting a notification prompting relearning. <P>SOLUTION: An appearance inspection device 100 includes a step (S310) of accepting input of an algorithm generation command, a step (S400) of executing algorithm generation processing, a step (S320) of accepting input of a threshold of ROC evaluation values and a condition of ROC analysis, a step (S330) of executing inspection using a generated algorithm, a step (S600) of executing ROC analysis in the case that the condition of ROC analysis is met (YES in a step 340), and a step (S370) of outputting information prompting relearning in the case that a ROC evaluation value meets a predetermined condition (YES in a step S360). <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a technique for inspecting an inspection object by executing image processing. More specifically, the present invention relates to a pattern recognition technique based on an algorithm generated by an optimization method, and more specifically, an appearance inspection apparatus, an appearance inspection method, and a computer having an image data learning function for inspection. The present invention relates to a program for functioning as an appearance inspection apparatus.

  Conventionally, some appearance inspection apparatuses automatically detect defects by performing image processing on image data obtained by imaging the appearance of an inspection target using an algorithm generated by an optimization method. For example, in an appearance inspection apparatus such as a semiconductor or a liquid crystal substrate, there is an example in which an image of defects such as cracks, hooks, foreign matter adhesion, and unevenness generated on the surface of the substrate is used as a learning image.

  In this image processing inspection process, when a defect pattern different from the input pattern assumed at the learning stage is input, the brightness of the illumination changes, and the captured image of the defect becomes totally or partially dark. The accuracy of image processing inspection may be reduced. Therefore, it is important to know the timing for re-learning the algorithm used for the image processing inspection by the optimization method. In order to solve these problems, techniques relating to image recognition or image processing are disclosed in Patent Documents 1 to 6.

  For example, Japanese Patent No. 3334807 (Patent Document 1) discloses a pattern classification apparatus that can determine the reliability of an output pattern in a neural network for pattern classification and can detect a decrease in the accuracy rate. This apparatus includes an input pattern monitoring module that compares a given input pattern with an input pattern of learning data used for learning of a neural network, a learning data file that stores learning data, and the above-described neural network for the given input pattern. An execution history monitoring module that compares the net output pattern with the correct output pattern for the input pattern, accumulates the comparison results as a history, and determines an abnormality from the comparison results, and an execution history that stores the history of the comparison results The file and the contents of the execution history file and the contents of the learning data file are read out, the contents of the read execution history file are additionally recorded in the learning data file, and compared with the execution history file. Create new learning data that does not detect Provided with a learning data update processing module that new to.

  According to the apparatus disclosed in Japanese Patent No. 3334807, the input pattern monitoring module compares the input pattern with the input pattern of each learning data used for learning of the neural network. Also, the input pattern and the distribution of the entire input pattern of the learning data are compared. That is, the position of the given input learning data in the distribution of the entire input pattern is calculated, and learning data having an input pattern similar to the given input pattern is searched. Such monitoring detects this fact if the input pattern given is not similar to any learning data input pattern. Thereby, since the user of the said apparatus can know that the reliability of a classification result is low with respect to the given input pattern, it can detect that a test | inspection precision falls.

  Japanese Laid-Open Patent Publication No. 10-171910 (Patent Document 2) discloses a diagnosis support apparatus that can efficiently detect an abnormality using an artificial neural network that has been learned using digital image data. The apparatus includes an extraction unit for extracting partial data from image data, a first neural network unit that outputs first signal data based on the partial data extracted by the extraction unit, and the first signal data A second neural network unit that outputs the second signal data based on the second signal data, and a determination unit for outputting a diagnosis result based on the second signal data.

According to the apparatus disclosed in Japanese Patent Laid-Open No. 10-171910, a receiver operating characteristic (ROC (Receiver Operating Characteristics)) is used to compare the superiority and inferiority of a generated algorithm (neural network) at the learning stage of a neural network. A technique is used in which learning is continued until an area below the curve becomes larger than a predetermined lower limit value.
Japanese Patent No. 3334807 Japanese Patent Laid-Open No. 10-171910 JP 2004-213567 A JP 2000-11093 A JP-A-6-237925 JP 2003-208595 A

  If the technique disclosed in Japanese Patent No. 3334807 is applied to the appearance of the inspection object based on image data obtained by photographing the inspection object, the following problems may occur. That is, when the gray value of an image pixel is used as the input pattern of the neural network, the dimension of the input pattern increases as the image size increases. For this reason, it is difficult to converge the learning of the neural network, which may cause a problem that it becomes impractical. Also, there may be a problem that the output result varies greatly depending on which part of the image has a defect. Furthermore, in the comparison between the input pattern and the input pattern used for learning, the entire image has a defect that only has a gray value offset (that is, a defect that looks similar), or the position has moved by a few pixels. When such image data is input, it may be determined that a different one is input. Therefore, there is a problem that it is not practical to use the gray value of the pixel of the image as the inspection input pattern.

  Therefore, it is conceivable to use a defect feature vector used for class determination in pattern recognition as an inspection input pattern. If the feature quantity vector is used, it is possible to some extent to judge the reliability of the inspection accuracy based on the distribution state of the defect feature quantity of the input pattern. However, Japanese Patent No. 3334807 discloses the optimization of classification / identification in pattern recognition, and does not disclose the contents directly related to the optimization of the image processing itself. Furthermore, since the index that requires re-learning is an average error between the output pattern of the neural network and the correct output pattern, quantitative evaluation cannot be performed. That is, there is a problem that a certain degree of experience is required to set the threshold for starting the relearning.

  On the other hand, according to the apparatus disclosed in Japanese Patent Application Laid-Open No. 10-171910, the horizontal direction profile of the inspection area image as the input of the first neural network, and the vertical direction of the inspection area image as the input of the second neural network. Using this profile, learning is continued until the area under the ROC curve becomes larger than a predetermined value. However, Japanese Patent Laid-Open No. 10-171910 does not disclose the start of relearning. Therefore, the diagnosis support apparatus has a problem that it cannot detect the timing for starting the relearning.

  The present invention has been made to solve the above-described problems. An object of the present invention is to provide an appearance inspection apparatus that probabilistically guarantees the reliability or accuracy of an inspection when an appearance inspection is performed using an algorithm generated by an optimization method.

  Another object of the present invention is to provide an appearance inspection apparatus capable of detecting the start timing of relearning.

  Another object of the present invention is to provide an appearance inspection method in which the reliability or accuracy of the inspection is probabilistically guaranteed when the appearance inspection is performed using an algorithm generated by an optimization method. .

  Another object of the present invention is to allow a computer to function as an appearance inspection apparatus that probabilistically guarantees the reliability or accuracy of the inspection when performing appearance inspection using an algorithm generated by an optimization method. Is to provide a program for

  In order to solve the above-described problems, according to one aspect of the present invention, an appearance inspection apparatus includes a generation unit that generates an algorithm for image processing, and input of image data generated by photographing a plurality of inspection objects. Receiving means. The appearance of each of the plurality of inspection objects varies depending on the time of shooting due to causes such as illumination deterioration and changes in optical conditions due to vibration. The appearance inspection apparatus includes a processing unit that performs image processing based on an algorithm for each image data generated by photographing each of the plurality of inspection objects, and each of the plurality of inspection objects based on the image data. Based on detection means for detecting defects, detection results by the detection means, and appearance inspection results prepared in advance for each of a plurality of inspection objects, the proportion of defects detected as defects, and non-defective products An evaluation value calculating means for calculating an evaluation value based on the ratio of detected as a defect, a determination means for determining whether the evaluation value and a predetermined threshold satisfy a predetermined condition, and a determination Based on the result of determination by the means, necessity determination means for determining whether or not it is necessary to newly generate an algorithm for image processing, and based on the result of determination by the necessity determination means. There are, and a re-learning means for generating a new algorithm for the generation means.

  Preferably, the determination unit determines whether or not the evaluation value is below a threshold value.

  Preferably, if the evaluation value and the threshold value do not satisfy the condition, the necessity determination unit determines that the defect pattern to be subjected to image processing is the defect class that was the target when the algorithm was generated. And the non-defective class are not included in the population, and it is determined that a new algorithm for image processing needs to be generated.

  Preferably, the visual inspection apparatus further includes storage means for storing image data used when it is determined that a new algorithm needs to be generated. The re-learning means causes the generating means to generate a new algorithm based on the image data used when it is determined that re-learning is necessary.

  Preferably, the evaluation value calculation means calculates an evaluation value based on feature amount data expressed by an appearance feature amount. The feature amount represents the feature of the image data generated by photographing. Whether the appearance inspection apparatus can acquire an evaluation value that satisfies a predetermined condition based on the discriminator data that is a reference for classifying the inspection object as either a defect or a non-defective product. It further includes an acquisition property determination means for determining. The relearning unit changes the value of the discriminator data by executing a predetermined analysis process based on the feature amount data.

  Preferably, the appearance inspection apparatus further includes notification means for notifying that a new algorithm needs to be generated based on the result of determination by the necessity determination means.

  Preferably, the appearance inspection device determines whether or not a defect has occurred on each surface of the plurality of inspection objects based on each image data generated by photographing each of the plurality of inspection objects. Means are further provided.

  Preferably, the relearning unit includes an acquisition unit for acquiring at least one of an algorithm for image processing and classifier data. The discriminator data is a standard for classifying each of the plurality of inspection objects as either a defect or a non-defective product. The appearance inspection apparatus further includes determination means for determining whether or not a defect has occurred on the surface of each of the plurality of inspection objects based on any data acquired by the acquisition means.

  According to another aspect of the present invention, an appearance inspection method includes a step of generating an algorithm for image processing and a step of receiving input of image data generated by photographing a plurality of inspection objects. The appearance of each of the plurality of inspection objects varies depending on the time of imaging. The visual inspection method includes a step of performing image processing based on an algorithm for each image data generated by photographing each of the plurality of inspection objects, and a defect in each of the plurality of inspection objects based on the image data , A detection result, and a ratio of a defect detected as a defect and a non-defective product detected as a defect based on a result of an appearance inspection prepared in advance for each of a plurality of inspection objects. An evaluation value based on the determined ratio, a first determination step for determining whether the evaluation value and a predetermined threshold satisfy a predetermined condition, and a first determination step Based on the result of the determination, a second determination step for determining whether or not it is necessary to newly generate an algorithm for image processing, and a determination in the second determination step. Based on the results, and generating a new algorithm.

  According to still another aspect of the present invention, the program is a program for causing a computer to function as an appearance inspection apparatus. The program causes a computer to execute an algorithm for image processing and a step of receiving input of image data generated by photographing a plurality of inspection objects. The appearance of each of the plurality of inspection objects varies depending on the time of imaging. The program performs a step of executing image processing based on an algorithm on each image data generated by photographing each of the plurality of inspection objects, and each of the plurality of inspection objects based on the image data. Based on the step of detecting the defect, the result of the detection, and the result of the appearance inspection prepared in advance for each of the plurality of inspection objects, the ratio of the defect detected as a defect and the non-defective product detected as a defect A step of calculating an evaluation value based on the ratio, a first determination step of determining whether the evaluation value and a predetermined threshold satisfy a predetermined condition, and a first determination step And a second determination step for determining whether or not a new algorithm for image processing needs to be generated, based on the determination result in Based on the results of determination by the flop, and a step of generating a new algorithm.

  According to the appearance inspection apparatus according to the present invention, the inspection accuracy and reliability can be calculated quantitatively. In addition, since the necessity of relearning is notified by setting a threshold value for inspection accuracy in advance, the appearance inspection apparatus can update learning data quickly.

  According to the program according to the present invention, the computer can function as an appearance inspection apparatus capable of quantitatively calculating inspection accuracy and reliability.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

  With reference to FIG. 1, an inspection system 10 including an appearance inspection apparatus according to an embodiment of the present invention will be described. FIG. 1 schematically shows a system configuration of the inspection system 10.

  The inspection system 10 includes a stage 111, a light source 113 that irradiates light on a predetermined range on the stage 111, a camera 114 that captures a predetermined range on the stage 111 and outputs an image signal, and a stage 111. It includes a controller 115 that controls operations of the light source 113 and the camera 114, respectively, and an appearance inspection apparatus 100 connected to the camera 114 and the controller 115. On the stage 111, an inspection object 112 to be inspected for appearance is mounted in a predetermined range. The appearance of the inspection object 112 varies depending on the timing of photographing by the camera 114 due to causes such as deterioration of illumination and changes in optical conditions due to vibration. The inspection object 112 is unloaded from the stage 111 when predetermined imaging processing and inspection processing are completed. The appearance inspection apparatus 100 includes an image processing unit 116, a main control unit 117, an auxiliary recording device 118, and an output unit 119.

  The image processing unit 116 performs predetermined image processing on the image signal output from the camera 114. The main control unit 117 controls the controller 115 by outputting a control signal for executing a predetermined operation. The controller 115 controls the operations of the stage 111, the light source 113, and the camera 114 based on the control signal. The main control unit 117 outputs a predetermined command to the image processing unit 116 to execute processing using an algorithm for image processing.

  The auxiliary recording device 118 stores data necessary for the operation of the appearance inspection device 100. As will be described later, this data includes data input from the outside and data generated by the main control unit 117 or the image processing unit 116. The output unit 119 outputs the state of the appearance inspection apparatus 100 based on the data generated by the main control unit 117. The output data includes, for example, a message indicating that learning in the appearance inspection apparatus 100 is necessary, and a message that prompts appearance processing using new parameters.

  The data structure of the appearance inspection apparatus 100 will be described with reference to FIG. FIG. 2 is a diagram conceptually showing one mode of data storage in the auxiliary recording device 118.

  The auxiliary recording device 118 includes areas 210 to 280. An algorithm generated in advance for executing image processing is stored in area 210. This algorithm is not limited to the one stored when the appearance inspection apparatus 100 is configured, and includes an algorithm updated by subsequent learning processing. An inspection result that is a result of executing predetermined image processing by the algorithm is stored in the area 220. That is, the result of the process executed based on the image data output from the camera 114 is stored in the area 220. On the other hand, the true result that is the result that should be output to the same inspection object 112 is stored in the area 230.

  The learning end condition is stored in the area 240. The termination condition includes, for example, the number of loop processes in a process to be described later, a learning time, or an evaluation value of learning. The result of the ROC analysis is sequentially stored every time data is generated. For example, “ROC analysis result 0001.data” stores the area 250. Thereafter, when the ROC analysis process is executed again, the results generated in response to the execution are accumulated. In the example shown in FIG. 2, for example, data generated every month is stored in areas 250 to 280, respectively.

  The data structure in the appearance inspection apparatus 100 is not limited to the aspect shown in FIG. For example, the auxiliary recording device 118 may be composed of a plurality of recording devices. Alternatively, any data shown in FIG. 2 may be stored in a removable recording medium.

  A control structure of appearance inspection apparatus 100 according to the present embodiment will be described with reference to FIGS. 3 to 6 are flowcharts showing the procedure of processes executed by the image processing unit 116 and the main control unit 117, respectively.

  Referring to FIG. 3, in step S310, main controller 117 receives an algorithm generation instruction input via an input interface (not shown). In step S400, the main control unit 117 executes algorithm generation processing to be described later. When this processing is executed, an algorithm for image processing is generated based on a predetermined optimization method.

  In step S320, main controller 117 receives an input of the ROC evaluation value threshold and ROC analysis conditions. Here, the ROC evaluation value refers to a value calculated based on the ROC curve. The ROC evaluation value will be described in detail with reference to FIG. The threshold value of the ROC evaluation value for performing relearning is set based on the input data. In addition, since it is necessary to grasp the ROC evaluation value at the time of learning, it is preferable to use the ROC evaluation value as the learning evaluation value in step S310, but other evaluation values may be used. For example, if ROC evaluation is performed using learning image data, it is possible to easily know the threshold value of the ROC evaluation value at the learning stage. Therefore, it is not necessary to use the ROC evaluation value as the learning evaluation value. In addition, a constant value may be set as the threshold value of the ROC evaluation value, or a condition that the ROC evaluation value falls by what percentage from the ROC evaluation value in the learning stage may be set. Furthermore, conditions for performing ROC evaluation are set. The condition may be, for example, an arbitrary timing of the user or may be periodic.

  In step S330, the main control unit 117 inspects the inspected object 112 using the generated algorithm. In step S340, main controller 117 determines whether or not the inspection result satisfies the condition for performing the ROC analysis. If main control unit 117 determines that the inspection result satisfies the condition (YES in step S340), the process proceeds to step S600. If not (NO in step S340), the process returns to step S330.

  In step S600, main controller 117 performs ROC analysis described later. When this process is executed, whether or not re-learning by the appearance inspection apparatus 100 is necessary is notified as the necessity.

  In step S360, main controller 117 determines whether or not the ROC evaluation value satisfies a predetermined condition. This determination is made based on, for example, whether the ROC evaluation value exceeds a predetermined threshold. If main control unit 117 determines that the ROC evaluation value satisfies the condition (YES in step S360), the process returns to step S330. If not (NO in step S360), the process proceeds to step S370.

  In step S370, main control unit 117 outputs information for prompting relearning to output unit 119. As a result, the output unit 119 outputs the information to the outside. Thereby, the user of the appearance inspection apparatus 100 can easily know that relearning is necessary. Thereafter, the process ends.

  FIG. 4 is a flowchart showing a processing procedure for generating an algorithm.

  In step S410, main controller 117 receives a learning image and a learning end condition via an input interface (not shown). For example, defect image data used for learning and pseudo defect image data are input. Note that the number of input images is not limited to one. As will be described later, in consideration of the accuracy of analysis using the ROC curve, it is desirable that there are many defect and non-defective sample images. The learning end condition includes, for example, the number of times the loop processing from step S420 to step S470 is executed, the learning time, the evaluation value for learning, the input of a stop command from the outside of the appearance inspection apparatus 100, and the like.

  In step S420, main controller 117 generates an algorithm for image processing using a predetermined optimization method. Optimization techniques include known techniques such as SA (Simulated Annealing), GA (Genetic Algorithms), and TS (Tabu Search), but may be other techniques. Such an optimization method is realized, for example, by executing a program that realizes necessary processing.

  In step S430, the main control unit 117 causes the image processing unit 116 to execute image processing using the algorithm. In step S440, main controller 117 extracts a feature amount from the result of image processing, and also extracts a feature amount vector. In step S450, the main control unit 117 classifies the image processing results using the feature amount, and executes processing for identifying the inspection results.

  In step S460, main controller 117 calculates a learning evaluation value based on a predetermined criterion. The evaluation value is calculated using, for example, the formula “class separation degree = inter-class variance / intra-class variance”. In addition, although the class separation degree is used for the learning evaluation value, other values may be used. For example, an ROC evaluation value may be used. In this case, it is necessary to determine the position of the classifier used for the actual inspection, that is, the value of the classifier on the distribution plane. For this determination, for example, a discriminant analysis method can be used.

  However, in the present embodiment, the dimension of the feature quantity is converted to one dimension by principal component analysis on the information of the two-dimensional feature quantity space. However, the two-dimensional feature quantity space is converted without converting the dimension of the feature quantity. As a discriminator, the shape of a curve expressed based on two fixed straight lines, a plurality of straight lines, a hyperbola, or a higher-order polynomial may be used. The learning evaluation value in this case can be calculated, for example, by using the above formula as a two-dimensional vector.

  In step S470, main controller 117 determines whether or not a learning end condition is satisfied. If main controller 117 determines that the condition is satisfied (YES in step S470), the process returns to step S420. If not (NO in step S470), the process ends and returns to the main process (FIG. 3). The image processing algorithm and the shape of the classifier are determined by the optimization method as described above.

  FIG. 5 is a flowchart showing details of the ROC analysis process.

  In step S510, main controller 117 receives an input of image data. In step S330, the main control unit 117 performs processing for inspection using the algorithm generated in the algorithm generation processing (S400).

  In step S340, main controller 117 determines whether or not the inspection result satisfies the ROC analysis condition. If main control unit 117 determines that the inspection result satisfies the condition (YES in step S340), the process proceeds to step S600. If not (NO in step S340), the process proceeds to step S550. In step S550, main controller 117 outputs the inspection result to output unit 119. In step S600, main controller 117 performs ROC analysis described later.

  FIG. 6 is a flowchart showing a processing procedure for ROC analysis.

  In step S610, main controller 117 receives an input of the inspection result and the correct answer result. Here, the correct answer result is a determination result of a true defect, for example, data obtained by visual inspection after an appearance inspection by an inspector.

  In step S620, main controller 117 derives data for representing the relationship between the defect recognition rate and the overdetection rate as an ROC curve. In step S630, main controller 117 calculates an ROC evaluation value.

  In step S640, main controller 117 determines whether or not the ROC evaluation value is equal to or less than a threshold value. If main controller 117 determines that the ROC evaluation value is equal to or less than the threshold (YES in step S640), the process proceeds to step S650. If not (NO in step S640), the process proceeds to step S660.

  In step S650, main control unit 117 generates data for notifying the ROC evaluation result, and outputs the data to output unit 119. As a result, the output unit 119 notifies the ROC evaluation result and indicates that relearning is necessary. Here, the appearance inspection apparatus 100 can indicate the direction of relearning. For example, the main control unit 117 performs principal component analysis again based on the data of the feature amount space used for the ROC evaluation, and optimizes the discriminator. The main control unit 117 determines whether or not the ROC evaluation value is improved. If the ROC evaluation value is improved, the main control unit 117 does not change the image processing algorithm, but uses the newly optimized discriminator to perform image processing and appearance inspection via the output unit 119. Notify Thereby, the user can determine whether or not to use the algorithm based on the ROC evaluation value. Note that this determination need not be made artificially. For example, the main control unit 117 may make a determination using the threshold value of the ROC evaluation value set in step S320 shown in FIG.

  On the other hand, when the desired ROC evaluation value cannot be obtained only by the optimization of the discriminator, the main control unit 117 re-learns the image processing algorithm and the discriminator by the optimization method. For example, the main control unit 117 executes the process for re-learning until an optimal solution is obtained by using an algorithm previously used for the inspection as an initial value.

  In step S660, main control unit 117 generates data for notifying the ROC evaluation result, and further outputs the data to output unit 119. As a result, the output unit 119 notifies the ROC evaluation result and displays that there is no need for relearning.

  With reference to FIG. 7, the relationship between the feature amounts extracted by the appearance inspection apparatus 100 will be described. FIG. 7 is a diagram illustrating a distribution state of feature quantity vectors.

  Here, in order to simplify the description, a case where a two-dimensional feature amount is used will be described. Note that one-dimensional feature values or two-dimensional or more feature values may be used instead of the mode shown in FIG. Here, the feature amount includes, for example, a pattern area, contrast, length, ferret diameter, frequency component, and the like. Distribution 710 represents, for example, a non-defective class distribution. Distribution 720 represents the distribution of defect classes. The first principal component 730 is calculated using main component analysis described later. That is, the first main component is

It is calculated by. The feature amount extracted in step S440 is projected perpendicularly to Equation 1 above. By such projection, the two-dimensional feature value is converted into a one-dimensional feature value.

  Here, a mode in which the two-dimensional feature value shown in FIG. 7 is converted into a one-dimensional feature value will be described with reference to FIG. FIG. 8 shows a probability density distribution. That is, the distribution 710 and the distribution 720 are each expressed on the first principal component axis in the form of probability density. The discriminator 800 can change the setting by moving on the first principal component axis, and can be an object of optimization as necessary. Here, the discriminator is a determination criterion for classifying whether or not the candidate is a true defect with respect to a defect candidate detected from image data obtained by photographing the inspection object 112.

  With reference to FIG. 9, the determination in the ROC analysis by the appearance inspection apparatus 100 according to the present embodiment will be described. FIG. 9 is a diagram showing the relationship between the defect recognition rate and the overdetection rate as an ROC curve.

  The over-detection rate represents a ratio at which it is determined as a defect that an image should be determined as a non-defective product as a result of image processing, that is, an erroneous determination ratio. The defect recognition rate represents a ratio of candidates that are determined to be defects, that is, a ratio that is correctly recognized.

  The ROC curve shown in FIG. 9 is for evaluating a portion (shaded portion) where the defect class (distribution 720) and non-defective product class (distribution 710) intersect in the distribution shown in FIG. As shown in FIG. 8, the data for drawing the ROC curve is acquired by moving the discriminator 800 within the shaded area. The ROC curve is originally used as an index for determining the superiority or inferiority of the algorithm. However, as shown in the present embodiment, when the input image changes, that is, the feature amount of the good product, the defect pattern, and the good product pattern. It may be used as an index indicating whether the distribution status has changed.

  Note that the ROC evaluation value in the present embodiment refers to the ratio of the area of the region 900 shown in FIG. 9 to the area of the triangle AOB, but is not limited thereto.

  With reference to FIG. 10, the output mode of the appearance inspection apparatus 100 will be described. FIG. 10 is a diagram illustrating a display mode of the examination result displayed on the output unit 119 and the necessity of relearning. Such a display is realized when the output unit 119 is a display panel or a display device, for example.

  The output unit 119 includes areas 1010 to 1030 for displaying the inspection result and the necessity of relearning based on the processing result in the appearance inspection apparatus 100. The result of the ROC evaluation is displayed in, for example, area 1010. The presence / absence of the need for re-learning is displayed in, for example, area 1020. Further, a message for prompting other processing based on the evaluation result is displayed in, for example, area 1030. These displays are realized based on message data stored in the auxiliary recording device 118 in advance. The output mode by the output unit 119 is not limited to that shown in FIG.

  The appearance inspection apparatus 100 according to the present embodiment as described above may be realized by hardware by combining circuits for realizing each process, or a program for realizing each process may be a CPU or the like. It can also be realized by software by causing the arithmetic unit to execute the above.

  Here, with reference to FIG. 11, a computer system 1100 that realizes the appearance inspection apparatus 100 according to the present embodiment will be described. FIG. 11 is a block diagram showing a hardware configuration of computer system 1100.

  The computer system 1100 includes a CPU 1110 connected to each other via a data bus, a mouse 1120 and a keyboard 1130 for receiving data input from the outside, data to be input or data temporarily generated by executing a predetermined process A volatile storage RAM (Random Access Memory) 1140, a hard disk 1150 capable of storing a large amount of data in a nonvolatile manner, a CD-ROM (Compact Disk-Read Only Memory) drive device 1160, a monitor 1180, Communication IF (Interface) 1190. A CD-ROM 1162 is attached to the CD-ROM drive device 1160.

  Processing in the computer system 1100 functioning as the appearance inspection apparatus 100 is realized by each hardware and software executed by the CPU 1110. Such software may be stored in the hard disk 1150 in advance, or may be stored and distributed in the CD-ROM 1162 or other storage medium, and read from the storage medium by the CD-ROM drive 160 or other reading device. In some cases, the data is once stored in the hard disk 1150. Alternatively, such software may be downloaded via the communication IF 1190 connected to the communication line. The software is read from the hard disk 1150 to the RAM 1140 and executed by the CPU 1110. The hardware itself of the computer system 1100 shown in FIG. 11 is general. Therefore, it can be said that the essential part of the present invention is software stored in the RAM 1140, the hard disk 1150, the CD-ROM 1162, and other storage media. Since the operation of each hardware of computer system 1200 is well known, detailed description will not be repeated.

  With reference to FIG. 12, a functional configuration of a computer system 1100 that realizes the appearance inspection apparatus 100 will be described. FIG. 12 is a diagram illustrating a configuration of functions realized by CPU 1110 by execution of software.

  The CPU 1110 includes a control unit 1200, an input / output unit 1210, a generation unit 1220, a processing unit 1230, a calculation unit 1240, a determination unit 1250, a necessity determination unit 1260, a learning unit 1270, and a notification data generation unit. 1280.

  The input / output unit 1210 executes data input / output via the data bus. The generation unit 1220 generates an algorithm for image processing using a predetermined optimization method. The processing unit 1230 executes the image processing on the image data generated by photographing the inspection object by using the algorithm. The calculation unit 1240 performs ROC analysis based on the result of the image processing and calculates an evaluation value. Determination unit 1250 determines whether the evaluation value and a predetermined threshold satisfy a predetermined condition. The necessity determination unit 1260 determines whether learning is necessary based on the determination result of the determination unit 1250. The learning unit 1270 executes processing for learning image data. The control unit 1200 causes the learning unit 1270 to perform relearning based on the determination result by the necessity determination unit 1260.

  Preferably, the predetermined condition is that the evaluation value falls below the threshold value.

  Preferably, if the evaluation value and the threshold value do not satisfy the above conditions, the necessity determination unit 1260 determines that the defect pattern that is the target of the image processing is the defect that was the target at the time of learning. It is judged that it is not included in the population of the class and the non-defective class, and it is judged that re-learning is necessary.

  Preferably, the control unit 1200 causes the learning unit 1270 to perform relearning using image data that has been used when it is determined that relearning is necessary.

  Preferably, the calculation unit 1240 performs ROC analysis based on feature amount data in a space expressed by the feature amount. The feature amount represents the feature of the image data generated by photographing. Determination unit 1250 determines whether or not an evaluation value that satisfies the predetermined condition can be acquired based on the discriminator data. The control unit 1200 changes the discriminator data by executing a predetermined analysis process based on the feature amount data.

  The learning unit 1270 performs learning using the image data used when it is determined that learning is necessary, for example. The learning unit 1270 changes the value of the discriminator by executing a predetermined analysis process based on the feature amount data.

  Preferably, CPU 1110 further includes a notification data generation unit 1280. The notification data generation unit 1280 generates data for notifying that learning is necessary based on the result of determination by the necessity determination unit 1260. Notification data generation unit 1280 generates, for example, data indicating the result of ROC analysis and a message indicating the necessity of learning. The generated data is sent to the monitor 1280 by the input / output unit 1210.

  Preferably, CPU 1110 further includes a defect determination unit 1290. The defect determination unit 1290 determines whether a defect has occurred on the surface of the inspection object based on the image data generated by photographing the inspection object.

  Preferably, the control unit 1200 executes processing for acquiring at least one of an algorithm for image processing and classifier data. The defect determination unit 1290 determines whether a defect has occurred on the surface of the inspection target based on any data acquired by the control unit 1200.

  As described above, the appearance inspection apparatus 100 according to the embodiment of the present invention can quantitatively know the inspection accuracy and reliability at intervals intended by the user of the apparatus. In addition, since the necessity of relearning is notified by setting a threshold value for inspection accuracy in advance, the learning data of the appearance inspection apparatus 100 can be quickly updated.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

It is a figure showing roughly the system configuration of inspection system 10 provided with the appearance inspection device concerning an embodiment of the invention. 3 is a diagram conceptually showing one mode of data storage in an auxiliary recording device 118 provided in the appearance inspection apparatus 100. FIG. 5 is a flowchart (part 1) illustrating a procedure of processes executed by the appearance inspection apparatus. It is a flowchart (the 2) showing the procedure of the process which the external appearance inspection apparatus 100 performs. It is a flowchart (the 3) showing the procedure of the process which the external appearance inspection apparatus 100 performs. 12 is a flowchart (No. 4) showing a procedure of processes executed by the appearance inspection apparatus 100. It is a figure showing the condition of distribution of the feature-value vector which the external appearance inspection apparatus 100 extracted. It is a figure showing distribution of probability density. It is a figure showing the relationship between a defect recognition rate and an overdetection rate as a ROC curve about the determination result in the ROC analysis by the appearance inspection apparatus 100. It is a figure showing the display mode of the inspection result displayed in the output part 119 with which the external appearance inspection apparatus 100 is provided, and the necessity of re-learning. 2 is a block diagram illustrating a hardware configuration of a computer system 1100 that implements an appearance inspection apparatus 100. FIG. It is a figure showing the structure of the function implement | achieved by execution of software by CPU1110 with which the computer system 1100 is provided.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 Inspection system, 100 Appearance inspection apparatus, 111 stage, 112 Inspected object, 113 Light source, 114 Camera, 115 Controller, 116 Image processing part, 117 Main control part, 118 Auxiliary recording apparatus, 119 Output part, 1100 Computer system, 1110 CPU, 1120 mouse, 1130 keyboard, 1140 RAM, 1150 hard disk, 1160 CD-ROM drive, 1162 CD-ROM, 1190 communication IF, 1180 monitor.

Claims (10)

Generating means for generating an algorithm for image processing;
Input means for receiving input of image data generated by photographing a plurality of inspection objects, the appearance of each of the plurality of inspection objects fluctuates according to the time of photographing,
Processing means for performing image processing based on the algorithm for each image data generated by photographing each of the plurality of inspection objects;
Detecting means for detecting a defect in each of the plurality of inspection objects based on the image data;
Based on the result of detection by the detection means and the result of the appearance inspection prepared in advance for each of the plurality of inspection objects, the ratio at which the defect is detected as a defect and the non-defective product is detected as a defect. An evaluation value calculating means for calculating an evaluation value based on the ratio,
Determining means for determining whether the evaluation value and a predetermined threshold satisfy a predetermined condition;
Necessity determining means for determining whether or not it is necessary to newly generate an algorithm for the image processing based on a result of determination by the determining means;
An appearance inspection apparatus comprising: a re-learning unit that causes the generation unit to generate a new algorithm based on a result of determination by the necessity determination unit.   The appearance inspection apparatus according to claim 1, wherein the determination unit determines whether the evaluation value is lower than the threshold value.   When the evaluation value and the threshold value do not satisfy the condition, the necessity determination unit is configured such that the defect pattern to be subjected to the image processing is targeted when the algorithm is generated. The visual inspection apparatus according to claim 1, wherein it is determined that the defect class and the non-defective class are not included in a population, and it is determined that a new algorithm for the image processing needs to be generated. Storage means for storing image data used when it is determined that the algorithm needs to be newly generated;
The appearance inspection apparatus according to claim 1, wherein the re-learning unit causes the generation unit to generate the new algorithm based on image data used when it is determined that the re-learning is necessary. The evaluation value calculation means calculates the evaluation value based on feature amount data expressed by the feature amount of the appearance, and the feature amount represents a feature of image data generated by the photographing,
Whether the appearance inspection apparatus can acquire an evaluation value that satisfies the predetermined condition based on discriminator data that is a reference for classifying the inspection object as either a defect or a non-defective product It further includes an acquisition property judging means for judging whether or not,
The appearance inspection apparatus according to claim 1, wherein the relearning unit changes a value of the discriminator data by executing a predetermined analysis process based on the feature amount data.   The appearance inspection apparatus according to claim 1, further comprising notification means for notifying that a new algorithm needs to be generated based on a result of determination by the necessity determination means.   The apparatus further comprises defect determination means for determining whether a defect has occurred on the surface of each of the plurality of inspection objects based on each image data generated by photographing each of the plurality of inspection objects. Item 1. An appearance inspection apparatus according to Item 1. The re-learning means includes acquisition means for acquiring data of at least one of an algorithm for image processing and discriminator data, and the discriminator data has a defect in each of the plurality of inspection objects. And a standard for classifying the product as a non-defective product,
The visual inspection apparatus further includes a determination unit that determines whether a defect has occurred on the surface of each of the plurality of inspection objects based on any data acquired by the acquisition unit. 1. An appearance inspection apparatus according to 1. Generating an algorithm for image processing;
Receiving the input of image data generated by photographing a plurality of inspection objects, the appearance of each of the plurality of inspection objects fluctuates according to the time of the photographing,
Performing image processing based on the algorithm for each image data generated by photographing each of the plurality of inspection objects;
Detecting defects in each of the plurality of inspection objects based on the image data;
Based on the result of the detection and the result of the appearance inspection prepared in advance for each of the plurality of inspection objects, the ratio at which the defect is detected as a defect and the ratio at which the non-defective product is detected as a defect Calculating an evaluation value based on
A first determination step of determining whether or not the evaluation value and a predetermined threshold satisfy a predetermined condition;
A second determination step of determining whether or not it is necessary to newly generate an algorithm for the image processing based on a result of the determination in the first determination step;
And a step of generating a new algorithm based on a result of the determination in the second determination step. A program for causing a computer to function as an appearance inspection apparatus, wherein the program is
Generating an algorithm for image processing;
Receiving the input of image data generated by photographing a plurality of inspection objects, the appearance of each of the plurality of inspection objects fluctuates according to the time of the photographing,
Performing image processing based on the algorithm for each image data generated by photographing each of the plurality of inspection objects;
Detecting defects in each of the plurality of inspection objects based on the image data;
Based on the result of the detection and the result of the appearance inspection prepared in advance for each of the plurality of inspection objects, the ratio at which the defect is detected as a defect and the ratio at which the non-defective product is detected as a defect Calculating an evaluation value based on
A first determination step of determining whether or not the evaluation value and a predetermined threshold satisfy a predetermined condition;
A second determination step for determining whether or not it is necessary to newly generate an algorithm for the image processing based on a result of the determination in the first determination step;
Generating a new algorithm based on a result of the determination in the second determination step.

Images