JP6641195B2 - Optimization method, optimization device, program, and image processing device - Google Patents

Description

  The present invention relates to an optimization process using a genetic algorithm or a genetic programming.

  Building a system for performing visual inspection and the like using digital images is a very difficult task for those who do not have knowledge of image processing. Even a person who has a high level of knowledge about image processing requires considerable man-hours to acquire image processing for obtaining a desired detection result. Therefore, Patent Document 1 discloses a method using a genetic algorithm when adjusting parameter values in the image processing so that a result image created as a result of the image processing matches a target image. . Patent Document 2 discloses a method of generating an image processing sequence for obtaining a desired processing result by using genetic programming. Non-patent Documents 1 to 6 propose algorithms for detecting outliers for high-dimensional data.

JP 2008-89574 A JP 2009-151371 A

Hans-Peter Kriegel, 2 others, "Angle-Based Outlier Detection in High-dimensional Data", [online], [Searched February 19, 2016], Internet <URL: http: //www.dbs.ifi .lmu.de / Publikationen / Papers / KDD2008.pdf> Charu C. Aggarwal, one outsider, "Outlier Detection for High Dimensional Data", [online], [Searched February 19, 2016], Internet <URL: http://charuaggarwal.net/outl.pdf> Hans-Peter Kriegel, 3 others, "Outlier Detection in Axis-Parallel Subspaces of High Dimensional Data", [online], [Searched February 19, 2016], Internet <URL: http: //www.dbs. ifi.lmu.de/Publikationen/Papers/pakdd09_SOD.pdf> Charles Sutton and 2 others, "Feature Bagging: Preventing Weight Undertraining in Structured Discriminative Learning", [online], [Searched February 19, 2016], Internet <URL: https://people.cs.umass.edu /~mccallum/papers/ir402bags.pdf> Fabian Keller, 2 others, "HiCS: High Contrast Subspaces for Density-Based Outlier Ranking", [online], [Search February 19, 2016], Internet <URL: http: //www.ipd.uni- karlsruhe.de/~muellere/publications/ICDE2012.pdf> Hans-Peter Kriegel, 2 others, "Outlier Detection in Arbitrarily Oriented Subspaces", [online], [Searched February 19, 2016], Internet <URL: http://www.dbs.ifi.lmu.de /~zimek/publications/ICDM2012/COP.pdf>

  By the way, in optimization processing using a genetic algorithm or genetic programming, the randomness of mutation and crossover in generating the next generation of individuals enables the search range to be widened, but the evaluation value Unsuitable individuals with very low fitness are also likely to occur. In practice, it is necessary to calculate an evaluation value to determine whether or not each individual is inappropriate. Therefore, evaluation values for all individuals of each generation are calculated. Therefore, a very long calculation time is required until an individual estimated as the optimal solution is obtained.

  The present invention has been made in view of the above problems, and has as its object to reduce the operation time required to reach an optimal solution in an optimization process using a genetic algorithm or genetic programming.

  The invention according to claim 1 is an optimization method using a computer while using a genetic algorithm or a genetic programming, wherein a) a plurality of individuals expressing a plurality of conditions to be optimized are stored. B) obtaining an evaluation value for each of the plurality of individuals by an evaluation process determined according to the optimization target; and c) giving priority to the individual having the higher evaluation value among the plurality of individuals. Generating a plurality of new individuals by genetic operations including selection and mutation and crossover, and d) repeating the steps b) and c) until a termination condition is satisfied. Obtaining an individual estimated as an optimal solution, wherein the step d) is repeated for d1) the plurality of teacher individuals including the individual generated in the step c). Constructing a classifier by performing learning using the feature vector obtained from each of them and a class based on the evaluation value in each of the plurality of teacher individuals; d2) after the d1) step A step of classifying a plurality of new individuals generated in each step c) by the classifier to limit the individuals for which the evaluation values are obtained in the step b) following the step c). Prepare.

  The invention according to claim 2 is the optimization method according to claim 1, wherein the plurality of individuals respectively represent a plurality of types of image processing sequences, and in the evaluation process, a predetermined target image is Thus, the degree of coincidence between the processed image generated by executing the image processing sequence represented by each individual and the target image to be generated from the target image is obtained.

  The invention according to claim 3 is the optimization method according to claim 2, wherein each individual is an element array in which a plurality of elements are arranged, and the plurality of elements are included in an image processing sequence. And a plurality of feature values in the feature vector are values of the plurality of parameters.

  The invention according to claim 4 is the optimization method according to claim 2 or 3, wherein, in the evaluation processing, an image processing sequence in which each individual expresses a plurality of target images including the target image. Is performed, and the degree of coincidence between the plurality of processed images generated by executing the plurality of target images and the plurality of target images to be generated from the plurality of target images is obtained. The classification result by the classifier in the step d2) is obtained. A class in which the evaluation value is obtained from a first number of target images in the next b) step, a class in which the evaluation value is obtained from a second number of target images smaller than the first number, And a class for which the evaluation value is not determined.

  The invention according to claim 5 is the optimization method according to any one of claims 1 to 4, wherein in the d2) step and each of the c) steps, the classifier is assigned to a class in which the evaluation value is obtained. The generation of a new individual is repeated until the number of individuals classified by is equal to the set number of individuals.

  The invention according to claim 6 is the optimization method according to any one of claims 1 to 5, wherein in step d), when a predetermined condition is satisfied, the classifier in step d1) is The construction is performed again.

  The invention according to claim 7 is an optimization device using a genetic algorithm or a genetic programming, wherein the storage unit stores a plurality of individuals each expressing a plurality of conditions to be optimized, and By an evaluation process determined according to the object to be converted, an evaluation value calculation unit that obtains an evaluation value in each of the plurality of individuals, a selection in which the high evaluation value of the plurality of individuals is preferentially included, and , An individual generation unit that generates a plurality of new individuals by a genetic operation including mutation and crossover, and processing in the evaluation value calculation unit and the individual generation unit is repeated until an end condition is satisfied, thereby obtaining an optimal solution. A repetition control unit that obtains an individual estimated as, wherein the repetition control unit is configured to execute the repetition control unit from each of the plurality of teacher individuals including the individual generated by the individual generation unit. A classifier constructing unit that constructs a classifier by performing learning using the obtained feature vector and a class based on the evaluation value in each of the plurality of teacher individuals; A classification control unit configured to classify a plurality of new individuals with the classifier and to limit individuals for which the evaluation value is calculated in the evaluation value calculation unit.

  The invention according to claim 8 is the optimization device according to claim 7, wherein the plurality of individuals respectively represent a plurality of image processing sequences, and in the evaluation processing, a predetermined target image Thus, the degree of coincidence between the processed image generated by executing the image processing sequence represented by each individual and the target image to be generated from the target image is obtained.

  The invention according to claim 9 is the optimization device according to claim 8, wherein each individual is an element array in which a plurality of elements are arranged, and the plurality of elements are included in an image processing sequence. And a plurality of feature values in the feature vector are values of the plurality of parameters.

  The invention according to claim 10 is the optimization device according to claim 8 or 9, wherein, in the evaluation processing, an image processing sequence in which each individual expresses a plurality of target images including the target image. Are performed, and the degrees of coincidence between the plurality of processed images generated by executing the plurality of target images and the plurality of target images to be generated from the plurality of target images are obtained. The classification result by the classifier is the evaluation value calculation. A class in which the evaluation value is obtained from a first number of target images, a class in which the evaluation value is obtained from a second number of target images smaller than the first number, and the evaluation value. Not including classes.

  The invention according to claim 11 is the optimization device according to any one of claims 7 to 10, wherein, after the classifier is constructed, the individual generation unit sets the class in which the evaluation value is determined. The generation of a new individual is repeated until the number of individuals classified by the classifier reaches the set number of individuals.

  According to a twelfth aspect of the present invention, in the optimization apparatus according to any one of the seventh to eleventh aspects, when a predetermined condition is satisfied, the iterative control unit classifies the classifier into the classifier constructing unit. Rebuild the container again.

  The invention according to claim 13 is a program for causing a computer to perform an optimization process using a genetic algorithm or a genetic programming, wherein the execution of the program by the computer causes the computer to: A step of storing a plurality of individuals each expressing a plurality of conditions; b) a step of obtaining an evaluation value for each of the plurality of individuals by an evaluation process determined according to the optimization target; Generating a plurality of new individuals by a selection in which the individuals with the higher evaluation values are preferentially included among the plurality of individuals, and a genetic operation including mutation and crossover; d) the steps b) and c) A) obtaining an individual estimated as an optimal solution by repeating the process until the termination condition is satisfied. Is learned using d1) a feature vector obtained from each of a plurality of teacher individuals including the individual generated in the step c) repeated in the step c), and a class based on the evaluation value in each of the plurality of teacher individuals. And a step d2) of classifying a plurality of new individuals generated in each step c) after the step d1) by the classifier. Limiting the individuals for which the evaluation value is determined in the step b) following the step.

  According to a fourteenth aspect of the present invention, there is provided the image processing apparatus, wherein the image storage unit stores a captured image of the target object, and the image processing sequence is acquired. An image processing unit that executes the image processing sequence on the captured image.

  ADVANTAGE OF THE INVENTION According to this invention, in the optimization processing using a genetic algorithm or a genetic programming, the operation time until it reaches the optimal solution can be shortened.

FIG. 2 is a block diagram illustrating a configuration of an inspection device. FIG. 2 is a diagram illustrating a configuration of a computer. FIG. 2 is a block diagram illustrating a functional configuration of the image processing acquisition device. FIG. 4 is a diagram for explaining an image processing sequence. It is a figure which shows the flow of a process which acquires the image processing sequence for defect area extraction. FIG. 14 is a diagram illustrating a change in the degree of coincidence in the processing of the comparative example. It is a figure showing an individual in genetic programming.

  FIG. 1 is a block diagram showing a configuration of an inspection apparatus 1 according to one embodiment of the present invention. The inspection device 1 is a device that inspects the appearance of various objects. Examples of an object to be inspected by the inspection apparatus 1 include a substrate such as a semiconductor substrate, a glass substrate, and a printed wiring board.

  The inspection device 1 includes an imaging unit 11, a defect detection unit 12, and a computer 2. The defect detection unit 12 includes an image storage unit 13 and an image processing unit 14. The imaging unit 11 captures an image of an object and acquires a captured image showing the appearance of the object. The captured image is input to the defect detection unit 12 and stored in the image storage unit 13. In the image processing unit 14, an image processing sequence for extracting a defective area acquired by the computer 2 is set, and the image processing sequence is executed on a captured image. Thus, when the target object includes a defect, an image indicating a defective area in the captured image is obtained. That is, a defect of the object is detected. In the processing by the image processing unit 14, a reference image described later may be used. In the inspection device 1, an image processing device is realized by the computer 2 that acquires the image processing sequence and the defect detection unit 12 that executes the image processing sequence. The defect detection unit 12 may be realized by the computer 2.

  FIG. 2 is a diagram illustrating a configuration of the computer 2. The computer 2 has a general computer system configuration including a CPU 21 for performing various arithmetic processes, a ROM 22 for storing basic programs, and a RAM 23 for storing various information. The computer 2 includes a fixed disk 24 for storing information, a display 25 for displaying various information such as images, a keyboard 26a and a mouse 26b for receiving input from an operator (hereinafter, collectively referred to as an "input unit 26"). A read / write device 27 that reads information from and writes information to a computer-readable recording medium 91 such as an optical disk, a magnetic disk, and a magneto-optical disk, and a communication unit 28 that communicates with the outside. Further included.

  In the computer 2, the program 92 is read in advance from the recording medium 91 via the read / write device 27 and stored in the fixed disk 24. The CPU 21 executes arithmetic processing according to the program 92 while using the RAM 23 and the fixed disk 24. Thereby, the function of the image processing acquisition device described later is realized.

  FIG. 3 is a block diagram illustrating a functional configuration of the image processing acquisition device 20 realized by the computer 2. The image processing acquisition device 20 includes a calculation unit 31 and a storage unit 32. The calculation unit 31 includes an individual generation unit 311, an individual storage unit 312, an evaluation value calculation unit 313, and a repetition control unit 314. The repetition control unit 314 includes a classification control unit 315 and a classifier construction unit 316. Details of the processing of the arithmetic unit 31 will be described later. The function of the arithmetic unit 31 may be constructed by a dedicated electric circuit, or a dedicated electric circuit may be partially used.

  The storage unit 32 stores a plurality of image data sets 40. Each image data set 40 includes target image data 41, reference image data 42, and target image data 43. The target image data 41 indicates a target image, the reference image data 42 indicates a reference image, and the target image data 43 indicates a target image.

  The target image is an image indicating a defect on the target object and its surroundings, and is acquired by, for example, the imaging unit 11. The reference image is an image indicating an object having no defect. The reference image indicates the same area as the target image. The target image is an image showing only a defective area in the target image. The target image is generated and prepared in advance, for example, by the operator performing an operation of painting a non-defective area in the target image. Of course, the target image may be generated by another method. The target image, the reference image, and the target image may be any of a multi-tone image or a binary image (the same applies to a processed image described later). The plurality of image data sets 40 include target images having different types of defects, different imaging conditions, and the like.

  In the image processing acquisition device 20, each target image is approximated to a target image included in the same image data set 40 by performing a process of executing a plurality of selected image processes on the images in a predetermined order as an image processing sequence. An image processing sequence to be performed is obtained. That is, an image processing sequence for extracting a defective area is acquired with the defective area as the extraction target area. The acquisition of the image processing sequence uses an optimization process described below using a genetic algorithm. The image processing acquisition device 20 is an optimization device that performs an optimization process with an image processing sequence for defect area extraction as an optimization target.

  FIG. 4 is a diagram for explaining the image processing sequence 5. In the image processing acquisition device 20, for example, the image processing sequence 5 is represented by the element array 6 shown in the lower part of FIG. 4, and the upper part of FIG. 4 abstracts the structure of the image processing sequence 5 shown by the element array 6. Is shown.

  The image processing sequence 5 includes a plurality of image processes sequentially executed. In the upper part of FIG. 4, a first image process 51a executed first in the plurality of image processes and a second image executed second in the plurality of image processes Processing 51b is shown. The image processing sequence 5 may include other image processing. In the image processing sequence 5, the target image data 41 is input, and the first image processing 51a is performed on the target image. Subsequently, the second image processing 51b is performed on the image on which the first image processing 51a has been performed. In this way, a plurality of image processes included in the image processing sequence 5 are sequentially executed, and the processing result of the last image processing is output as the processed image data 44. The processed image data 44 indicates a processed image.

  The element array 6 representing the image processing sequence 5 is an individual in the genetic algorithm. As shown in the lower part of FIG. 4, in the element array 6, a plurality of elements 600 each containing a value are arranged in a line. The plurality of elements 600 are assigned, for example, element numbers in ascending order. The element array 6 includes a first image processing unit 61a and a second image processing unit 61b. The first and second image processing units 61a and 61b correspond to the first and second image processing 51a and 51b in the image processing sequence 5, respectively. Each of the image processing units 61a and 61b is a set of at least one element 600. In the lower part of FIG. 4, the element 600 included in each of the image processing units 61a and 61b is surrounded by a thick broken-line rectangle. A plurality of elements 600 of the first image processing unit 61a indicate values of a plurality of parameters in the first image processing 51a. A plurality of elements 600 of the second image processing unit 61b indicate values of a plurality of parameters in the second image processing 51b. The parameters in each of the image processing 51a and 51b are, for example, the size of various filters (such as a noise removal filter) used in the image processing, the strength of the processing, or a threshold in the binarization processing.

  In executing the image processing sequence for the target image (obtaining the processed image), not only the target image but also other images may be used. For example, when one image process is a process of acquiring a difference image between a target image and a reference image and binarizing the difference image, the process is performed using the target image and the reference image. Further, when one image process is a process of acquiring a difference image between the target image subjected to one filter process and the reference image subjected to another one filter process and binarizing the difference image, The image processing includes parallel processing on the target image and the reference image. Furthermore, one image process and another image process may be performed in parallel, and another image process may be performed using two images acquired by both image processes.

  FIG. 5 is a diagram illustrating a flow of a process in which the image processing acquisition device 20 acquires an image processing sequence for extracting a defective area. In obtaining an image processing sequence, first, a plurality of image data sets 40 are prepared and input to the image processing obtaining device 20. The plurality of image data sets 40 are stored in the storage unit 32. Further, based on the input of the operator via the input unit 26, a plurality of element arrays 6 respectively representing a plurality of types of image processing sequences are generated and stored in the individual storage unit 312 (step S11). The plurality of element arrays 6 are different from each other, and indicate an image processing sequence under a plurality of conditions. As described above, the element array 6 is an individual in the genetic algorithm, and is hereinafter referred to as “individual 6”.

  In the plurality of individuals 6, the number of elements 600 is the same. In the plurality of individuals 6, the elements 600 having the same element number correspond to the same image processing parameter of the image processing sequence 5, and the value of the element 600 indicating the value of the parameter differs in principle. The plurality of individuals 6 belong to the initial generation in the genetic algorithm. Hereinafter, the number of individuals 6 prepared in step S11 is referred to as “set number of individuals”. In this processing example, the set number of individuals is two or more. In step S11, only the basic structure of the image processing sequence 5 is determined by the operator, and the individual generation unit 311 randomly determines the value of each element 600, so that a plurality of individuals 6 of the initial generation may be prepared. . The range of values that can be taken by the parameters of each image processing is set in advance.

  Subsequently, the evaluation value calculation unit 313 obtains an evaluation value for each of the plurality of individuals 6 by a predetermined evaluation process (step S12). In the evaluation process, by executing each of the image processing sequences of the set number of individuals indicated by the set number of individuals 6 on the target image, processed images of the set number of individuals are acquired for the target image. Then, a value indicating the degree of coincidence between each processed image and a target image to be generated from the target image is obtained. As the value indicating the degree of coincidence, for example, in a difference image between the binary processed image and the binary target image, an area ratio or the like of a region where the pixel value is 0 is obtained. The value indicating the degree of coincidence between the processed image and the target image may be obtained by various methods.

  As described above, in the image processing acquisition device 20, a plurality of image data sets 40 are prepared, and a plurality of processed images are acquired by the image processing sequence indicated by each individual 6. In the evaluation value calculation unit 313, a representative value of a value indicating the degree of coincidence in a plurality of processed images acquired using each individual 6 is obtained as an evaluation value (fitness) of the individual 6. Each individual 6 and the evaluation value are output to the repetition control unit 314 and stored in the classifier construction unit 316. Here, the representative value is a value indicating the vicinity of the center of the distribution of a plurality of values, and is typically an average value or a median value.

  When the respective evaluation values of the plurality of individuals 6 are obtained, the repetition control unit 314 checks whether the termination condition is satisfied (step S13). For example, when the largest evaluation value (hereinafter, referred to as “generation evaluation value”) among the evaluation values of the plurality of individuals 6 of the current generation (currently, the initial generation) is equal to or greater than a predetermined setting value, When the generation evaluation value does not change for a predetermined number of generations, or when the current generation is a generation number specified in advance, it is determined that the termination condition is satisfied. At this time, it is assumed that the termination condition has not been satisfied.

  In addition, the repetition control unit 314 checks whether or not the construction condition of the classifier is satisfied (step S14). In this processing example, the generation (generation number) in which the classifier should be constructed is predetermined, and when the current generation is the generation, it is determined that the construction condition of the classifier has been satisfied. At this point, it is assumed that the construction condition of the classifier is not satisfied.

  In the individual generation unit 311, the same number of individuals 6 (that is, the set number of individuals) as the current generation are selected as new individuals 6 from the plurality of individuals 6 in the current generation (step S <b> 15). The individuals 6 of the new set number of individuals may include the same individuals 6 of the current generation redundantly. The selection of a new individual 6 is performed based on the evaluation value of the individual 6 of the current generation. For example, a value obtained by dividing the evaluation value of each individual 6 of the current generation by the sum of the evaluation values of all the individuals 6 of the current generation is obtained as the selection probability of the individual 6. Is included in a new individual 6 (so-called roulette selection). As described above, in the selection of the individual 6 in step S15, the individual 6 having the higher evaluation value among the plurality of individuals 6 of the current generation is preferentially included in the new individual 6.

  Subsequently, a genetic operation is performed on the individuals 6 of the new set number of individuals. Specifically, first, mutation of the element 600 of each new individual 6 is performed (step S16). For example, whether or not to actually mutate each element 600 is determined according to a preset mutation probability. As described above, in each element 600 of the individual 6, the numerical range that the element 600 can take is set in advance, and the value of the element 600 that is determined to be mutated is within the possible numerical range. Is randomly changed to any one of

  When the mutation of each individual 6 is completed, crossover is performed to generate two new individuals 6 from the two individuals 6 (step S17). In this processing example, a plurality of combinations (pairs) of two individuals 6 are determined in the individuals 6 of the set number of individuals after mutation. Then, in each combination of the two individuals 6, the crossover range is determined at random, and a crossover (two-point crossover) is performed in which the elements 600 in the crossover range are exchanged between the two individuals 6. The determination of the crossover range may include a case where there is no crossover range, that is, a case where no crossover is performed. In the crossover, one-point crossover, uniform crossover, or the like may be performed.

  As described above, by the selection based on the evaluation value and the genetic operation including mutation and crossover, the set number of individuals 6 included in the new generation (currently the second generation) is generated, It is stored in the individual storage unit 312. In the generation of the plurality of individuals 6 of the new generation, elite storage or the like is performed in which the individual 6 having the largest evaluation value among the evaluation values acquired in the process of the immediately preceding step S12 is added to the new individual 6 as it is. May be. Further, the crossover in step S17 may be performed before the mutation in step S16.

  When the plurality of individuals 6 of the new generation are generated, the process returns to step S12, and the image processing sequence indicated by each individual 6 of the new generation is executed on the plurality of target images, so that a plurality of processes are performed. A completed image is obtained. Then, the evaluation value of the individual 6 is obtained based on the value indicating the degree of coincidence between the plurality of processed images and the corresponding plurality of target images. In a state where no classifier has been constructed, step S18 is skipped.

  The repetition control unit 314 checks whether the termination condition is satisfied and whether the classifier construction condition is satisfied (steps S13 and S14). When the termination condition and the construction condition of the classifier are not satisfied, the individual 6 is selected, mutated, and crossed over in the same manner as described above, and a new plurality of individuals 6 of the next generation is generated ( Steps S15 to S17). Then, an evaluation value of the plurality of individuals 6 is calculated by the evaluation process on the plurality of individuals 6 (step S12). In this way, the processing of steps S15 to S17 and S12 is repeated until the termination condition or the classifier construction condition is satisfied (steps S13 and S14).

  When the current generation is a generation for which a classifier should be constructed, and it is confirmed in step S14 that the construction condition of the classifier is satisfied, the classifier constructing unit 316 constructs the classifier 317 (step S19). In the construction of the classifier 317, for example, all the individuals 6 in all the generations up to the current generation are treated as teacher individuals. That is, the plurality of teacher individuals include the plurality of individuals 6 generated by the repeated steps S15 to S17. The generation for which the classifier is to be constructed can be considered as a generation in which the number of individuals 6 suitable for the construction of the classifier 317 is prepared. The generation for which a classifier is to be constructed may be a generation in which the rate of increase from the generation evaluation value of the immediately preceding generation in the generation evaluation value of each generation is equal to or less than a predetermined value.

  As described above, the plurality of elements 600 of each individual 6 indicate values of a plurality of parameters in a plurality of image processes included in the image processing sequence represented by the individual 6. By capturing the values of the plurality of parameters as a plurality of feature amounts, a feature vector defined by the plurality of feature amounts is obtained for each teacher individual. Also, the evaluation value of each teacher individual is stored. For example, a class of “adopted” (label) is assigned to a teacher individual having an evaluation value equal to or greater than the determination threshold with a certain value as a determination threshold, and Is assigned to the teacher individual having the evaluation value of “rejected”. In other words, the class of “adopted” or the class of “not adopted” is taught to each teacher individual. The determination threshold is, for example, a value that divides all teacher individuals arranged in descending order of evaluation values into a class of “adopted” and a class of “non-adopted” at a predetermined ratio, or a predetermined ratio of the maximum evaluation value in all teacher individuals. It is determined as a corresponding value or the like. Of course, the determination threshold may be input by the operator. The plurality of teacher individuals may include individuals prepared in advance by the operator.

  In the classifier constructing unit 316, the feature vector of each teacher individual is input to the classifier 317. Then, learning is performed so that the class classified by the classifier 317 is the same as the class of the teacher individual. In this way, the classifier 317 is constructed by using the feature vectors of the plurality of teacher individuals and the classes based on the evaluation values of the plurality of teacher individuals as the teacher data. Building a classifier means generating a classifier by assigning values to parameters included in the classifier, determining the structure, or the like. As a method (learning algorithm) or mechanism for constructing the classifier 317, for example, linear discriminant analysis, a decision tree method, a neural network, a support vector machine (SVM), logistic regression, or the like can be adopted.

  After the classifier 317 is constructed, when a plurality of individuals 6 of a new generation are generated in steps S15 to S17, the classification control unit 315 inputs the feature vector of each individual 6 of the new generation to the classifier 317. Thereby, the individual 6 is classified into either the “adopted” class or the “non-adopted” class (step S18). Then, in the next step S12, a plurality of processed images are obtained from the plurality of target images using only the individual 6 classified into the “adopted” class, and the evaluation value of the individual 6 is calculated. It is predicted that the evaluation value of the individual 6 classified into the “adopted” class will be a certain value or more. For the individual 6 classified into the “not adopted” class, no processed image is obtained, and no evaluation value is calculated.

  In calculating the evaluation value for the individual 6 classified into the “adopted” class, only a part of the target images may be used. For example, all target images are divided into a plurality of target image groups, and an evaluation value is calculated from target images included in one target image group. The target image group used for calculating the evaluation value is sequentially changed or randomly selected. In each generation, some target images used for calculating the evaluation value may be randomly extracted from all target images.

  As described above, after the step S19 is performed, every time the steps S15 to S17 are performed, the plurality of individuals 6 of the new generation are classified by the classifier 317 in the step S18, so that the next step S12 is performed. Are restricted (screened) for which the evaluation value is required. The processes of steps S15 to S18 and S12 are repeated until the termination condition is satisfied (step S13). At this time, in step S15 of selecting a new individual 6 based on the evaluation value of the individual 6 of the current generation, for example, a judgment is made on the individual 6 classified into the “non-adopted” class for which an evaluation value is not obtained. A constant evaluation value sufficiently lower than the threshold value (including 0, which is the minimum evaluation value) is given. Therefore, the individuals 6 classified into the “non-adopted” class are substantially eliminated.

  Classifier 317 may be updated. For example, in the case where the classifier construction condition includes updating of the classifier every fixed number of generations, every time the generation changes by the number of generations from the generation of the classifier 317 (every time the classifier construction condition is satisfied) 2), under the control of the repetition control unit 314, the classifier 317 is constructed again by the classifier construction unit 316 in step S19. Updating of the classifier 317 may be performed in each generation after the first classifier 317 is constructed.

  The plurality of teacher individuals used for the construction (update) of the second and subsequent classifiers 317 are, for example, evaluation values actually calculated for all individuals 6 in all generations up to the current generation. Individuals 6 classified into the “not adopted” class are not included. In the construction of the classifier 317, only the individuals 6 in the latest predetermined number of generations may be used as teacher individuals. In this case, the number of teacher individuals is relatively small, and the time required to construct the classifier 317 is reduced. Is done. The individual 6 (individual 6 whose evaluation value is not calculated) may be included in the teacher individual while the class of “non-adoption” is assigned to the individual 6 classified in the class of “non-adoption”. The condition for constructing the classifier may include a case where the generation evaluation value rises by a predetermined value from the generation evaluation value obtained when the immediately preceding classifier 317 was constructed.

  As described above, while updating the classifier 317 as necessary (steps S14 and S19), generation of a plurality of individuals 6 of a new generation (steps S15 to S17), classification of the plurality of individuals 6 (step S18) , And the calculation of the evaluation value (step S12) are repeated until the termination condition is satisfied (step S13). When it is confirmed in step S13 that the termination condition is satisfied, for example, among the plurality of individuals 6 of the current generation for which the evaluation value has been obtained in the immediately preceding step S12, It is determined as an individual 6 indicating an image processing sequence for extracting a defective area. That is, the individual 6 estimated as the optimal solution in the main optimization process using the genetic algorithm is obtained (step S20). The image processing sequence for defect area extraction is input to the image processing unit 14 of the defect detection unit 12, and is used for inspection of the appearance of the object in the inspection device 1.

  Next, a description will be given of the processing of the comparative example in which the construction of the classifier (steps S14 and S19) and the classification of the individual (step S18) are omitted in the processing of FIG. In the process of the comparative example, generation of a plurality of individuals of a new generation (steps S15 to S17) and calculation of evaluation values for all of the plurality of individuals (step S12) are repeated until the termination condition is satisfied (step S12). Step S13).

  FIG. 6 is a diagram illustrating a change in the degree of coincidence in the processing of the comparative example. In the processing of the comparative example, ten target images (image data sets 40) are used. In FIG. 6, the matching degree (matching degree of 20 individuals) obtained for three representative target images is shown. In the description with reference to FIG. 6, the same applies to the following.), Lines L1, L2, and L3 are shown, and the average of the degrees of coincidence in the ten target images is represented by a line L0. Shown by the attached line. As can be seen from FIG. 6, the degree of coincidence is sharply improved in the initial generation, but the degree of coincidence is gradually reduced as the generation increases. Actually, the degree of coincidence hardly changes over several tens of generations in some cases. In the processing of the comparative example, even in such a period, the evaluation values of all the individuals are calculated, and the number of times of executing the image processing sequence on the target image increases.

  Here, typical image processing included in the image processing sequence is processing using a filter. In this processing, a product-sum operation for each pixel of the target image is performed for the size of the filter. If the size of the filter is 3 rows and 3 columns and the size of the target image is 1000 rows and 1000 columns, (3 × 3) product-sum operations are repeated (1000 × 1000) times. Actually, since the image processing sequence includes a plurality of image processes, it takes a certain amount of calculation time to execute one image processing sequence for one target image. If the calculation time is about 1 second and it takes 30 generations to escape from the stagnation of evolution for one target image, 600 seconds (20 [individual] × 30 [generation] × 1 [second] ) Calculation time is wasted. When a plurality of target images are prepared, the calculation time further increases.

  On the other hand, in the image processing acquisition device 20 of FIG. 3, when a plurality of teacher individuals including the individuals 6 generated by the individual generation unit 311 are prepared, a feature vector acquired from each of the plurality of teacher individuals, A classifier 317 is constructed by performing learning using a class based on the evaluation value in each of the plurality of teacher individuals. Then, after constructing the classifier 317, the new plurality of individuals 6 generated by the individual generation unit 311 are classified by the classifier 317, whereby the individual 6 whose evaluation value is obtained by the evaluation value calculation unit 313 is obtained. Limited (limited).

  For example, when the value used for class classification in the classifier 317 is obtained by a product-sum operation of hundreds of feature amounts indicated by the feature vectors and weighting factors respectively corresponding to these feature amounts, In the classification of one individual 6, several hundred product-sum operations are performed only once. Therefore, the calculation time required for the classification of one individual 6 by the classifier 317 is extremely shorter than the calculation of the evaluation value for the individual 6 (execution of the image processing sequence for the target image). The calculation time required for classifying the individual 6 does not depend on the size of the target image used for calculating the evaluation value, the number of target images, the complexity of the image processing sequence, or the like.

  Therefore, in the image processing acquisition device 20 that limits the individual 6 for which the evaluation value is obtained by the classification using the classifier 317, the number of executions of the image processing sequence is reduced, and the calculation time until reaching the optimal solution is shortened. be able to. As a result, a preferable (highly accurate) image processing sequence for defect area extraction can be efficiently acquired in a short time. The defect detection unit 12 using the image processing sequence can accurately extract a defective area from a captured image.

  Further, a plurality of processed images are obtained from the plurality of target images by the image processing sequence indicated by each individual 6, and an evaluation value of the individual 6 is obtained based on the plurality of processed images. As a result, it is possible to stably acquire an image processing sequence capable of accurately extracting a defective area from various captured images. Note that only one target image may be used depending on the accuracy required for the image processing sequence for defect area extraction.

  In the image processing acquisition device 20, when the individuals 6 classified by the classifier 317 into the “non-adopted” class exist among the individuals 6 of the set number of individuals generated by the individual generation unit 311, the “non-adopted” A new individual 6 instead of the individual 6 may be further generated. That is, the generation of a new individual 6 by the individual generation unit 311 (Steps S15 to S17) and the classification of the individual 6 by the classifier 317 (Step S18) are classified into the “adopted” class for which an evaluation value is obtained. The process is repeated until the number of individuals 6 reaches the set number of individuals. Then, the evaluation value of the individual 6 of the set number of individuals classified into the “adopted” class is calculated by the evaluation value calculation unit 313 (step S12). Also in this case, since the evaluation process for the individual 6 classified into the “non-adopted” class is substantially omitted, the calculation time required to reach the optimal solution can be reduced. In addition, stagnation of evolution in the optimization processing is also suppressed.

  Next, another processing example in which the classifier 317 classifies the individual 6 into one of the three classes will be described. In the other processing example, in the construction of the classifier in step S19, a class of “good” is assigned to a teacher individual having an evaluation value equal to or greater than the first determination threshold, and is less than the first determination threshold, and A class “OK” is assigned to a teacher individual having an evaluation value equal to or higher than the second determination threshold value lower than the second determination threshold. In addition, a class of “not adopted” is assigned to a teacher individual having an evaluation value less than the second determination threshold. Then, the classifier constructing unit 316 constructs a classifier 317 by performing learning using feature vectors of the plurality of teacher individuals and classes based on the evaluation values of the plurality of teacher individuals.

  In step S18, each individual 6 of the new generation is classified by the classifier 317 into one of the "good" class, the "acceptable" class, and the "non-adopted" class. In the next step S12, no processed image is obtained for the individual 6 classified into the class of "not adopted", and no evaluation value is calculated. On the other hand, for the individuals 6 classified into the class “OK”, a predetermined first number of target images (for example, all target images) among a plurality of target images stored in the storage unit 32 are used. A first number of processed images is obtained. Then, the representative value of the value indicating the degree of coincidence in these processed images is calculated as the evaluation value of the individual 6. In addition, for the individual 6 classified into the “good” class, a second number of target images (for example, half target images) smaller than the first number is used for the second target image among the plurality of target images. Number of processed images are obtained. Then, the representative value of the value indicating the degree of coincidence in these processed images is calculated as the evaluation value of the individual 6. The generation of the new generation individual 6 based on the evaluation value (steps S15 to S17) is the same as described above.

  As described above, in the above-described processing example, the classification result of the classifier 317 in step S18 is obtained by comparing the class of “OK” for which the evaluation value is obtained from the first number of target images in the next step S12 with the first number The class includes a “good” class in which an evaluation value is obtained from a second smaller number of target images and a “non-adopted” class in which an evaluation value is not obtained. As described above, in the individuals 6 in the “good” class where a high evaluation value is expected, the calculation time can be further reduced by reducing the number of target images used for calculating the evaluation value. Of course, the classifier 317 may classify the individual 6 into any of four or more classes.

  Further, the classifier constructing unit 316 may construct a classifier 317 that performs outlier detection. In the construction of such a classifier 317, among all the individuals 6 in all generations up to the current generation, for example, an evaluation value of 70% or less of the maximum evaluation value (the maximum evaluation value of all the individual 6) is set to Only the possessed individual 6 is treated as a teacher individual. That is, the teacher individual is the individual 6 whose evaluation value is relatively low. Then, by performing learning using a plurality of feature vectors obtained from a plurality of teacher individuals, a classifier 317 that detects outliers for a relatively low set of evaluation values is constructed.

  As an algorithm that can be used for outlier detection using a feature vector that is high-dimensional data, a one-class support vector machine is exemplified. The method described in "Angle-Based Outlier Detection in High-dimensional Data" by Hans-Peter Kriegel et al., The method described in "Outlier Detection for High Dimensional Data" by Charu C. Aggarwal et al., The method described by Hans-Peter Kriegel et al. The method described in "Outlier Detection in Axis-Parallel Subspaces of High Dimensional Data", the method described in "Feature Bagging: Preventing Weight Undertraining in Structured Discriminative Learning" by Charles Sutton et al. The method described in "HiCS: High Contrast Subspaces for Fabian Keller" The method described in "Density-Based Outlier Ranking" and the method described in "Outlier Detection in Arbitrarily Oriented Subspaces" by Hans-Peter Kriegel et al. Can also be used (see Non-Patent Documents 1 to 6 above).

  In the classification control unit 315, the feature vector of each individual 6 generated by the individual generation unit 311 is input to the classifier 317. Then, the evaluation value is calculated by the evaluation value calculation unit 313 only for the individual 6 whose feature vector is detected as an outlier by the classifier 317. As described above, since the evaluation values of all the teacher individuals are relatively low, the individual 6 detected as an outlier by the classifier 317 is expected to obtain a high evaluation value. In this processing example, a class of “not adopted” is assigned to all the teacher individuals based on the evaluation value, and the individual 6 detected as an outlier by the classifier 317 substantially has the “adopted” class. It can be regarded as being classified into classes.

  Even when the classifier 317 that performs outlier detection is used, the number of individuals 6 to be subjected to the evaluation processing can be reduced, and the calculation time required to reach an optimal solution can be reduced. Also, the classifier 317 that performs outlier detection may be updated in the same manner as in the above processing example. In this case, typically, as the classifier 317 is updated, the maximum evaluation value of the teacher individual used for updating increases, and the classifier 317 that detects the individual 6 having a higher evaluation value as an outlier is used. Be built. That is, as the classifier 317 is updated, the performance of the classifier 317 improves.

  In the optimization processing in the image processing acquisition device 20, genetic programming may be used. In genetic programming, an individual expressing an image processing sequence has a tree structure. For example, the individual 6a illustrated in FIG. 7 includes terminal nodes I1 to I4 indicating input of one image and non-terminal nodes F1 to F7, F10, and F100 that receive one or more images and output one image. And The target image or the reference image (data of) included in one image data set 40 is input to the end nodes I1 to I4.

  Non-terminal nodes F1 to F7, F10, and F100 typically represent image processing filters. In one-input non-terminal nodes F1 to F7, predetermined image processing (average value filter processing, maximum value filter processing, or the like) is performed on one input image, and one image after processing is processed by the next node. Is output to In the two-input non-terminal node F10, predetermined image processing (logical addition operation, logical product operation, and the like) is performed on two images input from the two non-terminal nodes F3 and F5, and one image after the processing is performed. Is output to the next node. In the three-input non-terminal node (root node) F100, predetermined image processing is performed on three images input from the non-terminal nodes F6 and F7 and the terminal node I4, and one processed image is processed. Is output as In the image processing acquisition device 20, many types of image processing are prepared, and various image processing sequences are expressed by assigning arbitrary image processing as each node of various tree structures.

  In the optimization processing of FIG. 5, in the mutation in step S16, a node or a subtree is changed or deleted at random in the tree structure of each individual 6a. In the crossover in step S17, in each of the two individuals 6a forming a pair, one node is randomly selected as a crossover position, and a subtree that is a terminal part of the crossover position is replaced with the two tree structures. As a result, two new individuals 6a are generated. The selection of the individual 6a in step S15 and the calculation of the evaluation value in step S12 are the same as in the case of the genetic algorithm.

  In the construction of the classifier in step S19, in each teacher individual (tree structure), the number of nodes of three inputs, the number of nodes of two inputs, the number of nodes of input, the number of nodes indicating each type of image processing, and the like Is treated as a feature amount, and a feature vector is obtained from the teacher individual. Then, the learning is performed using the feature vector of each teacher individual and a class based on the evaluation value of the teacher individual, whereby the classifier 317 is constructed. In step S18, the individual 6a is classified into one of the classes by inputting the feature vector obtained from each individual 6a of the new generation to the classifier 317.

  As described above, the method of restricting the individual 6a for which the evaluation value is obtained by the classification using the classifier 317 can be adopted also in the optimization processing using the genetic programming. Also in this case, the number of times of the evaluation processing can be reduced, and the calculation time until reaching the optimum solution can be shortened.

  Various modifications are possible in the inspection device 1 and the image processing acquisition device 20.

  In the above embodiment, a common classifier 317 is constructed for a plurality of target images. However, depending on the design of the image processing acquisition device 20, different classifiers may be constructed for a plurality of target images. Good. Specifically, using the degree of coincidence between the processed image acquired by applying the image processing sequence represented by each teacher individual to each target image and the corresponding target image, The class is determined. In the classifier constructing unit 316, a classifier for each target image is constructed by learning using feature vectors of a plurality of teacher individuals and classes of the plurality of teacher individuals regarding the target image. When classifying each individual, for example, the individual is classified into a class of “adopted” or a class of “non-adopted” by majority decision of classification results by a plurality of classifiers.

  In the inspection apparatus 1, an image obtained by imaging EL (electroluminescence) emission or PL (photoluminescence) emission of a solar cell panel, or an image of a solar cell panel obtained by using a laser terahertz emission microscope (LTEM) The defect of the solar cell panel may be detected using the method described above. Further, a defect of the component may be detected using images of various components formed of resin, metal, or the like. The image used for detecting a defect may be an image captured by an electron beam, X-ray, or the like.

  The image processing acquisition device 20 may acquire an image processing sequence other than the image processing sequence for extracting a defective area. For example, an image processing sequence for extracting a cell region is obtained by the above-described optimization process, using a cell image obtained by imaging cells in a predetermined liquid such as blood or a culture solution as a target image and an area of the cell as an extraction target area. May be done. Further, an image processing sequence used for a purpose other than the extraction of the region may be obtained.

  The above optimization process using a genetic algorithm or genetic programming is used for optimization of conditions in various optimization targets, such as various product designs, process conditions, text and voice analysis algorithms, in addition to image processing sequences. It is possible to For example, when the design of a lens is to be optimized, the individual generation unit 311 generates a plurality of individuals expressing a plurality of conditions of a design parameter group such as a radius of curvature, a thickness, and a refractive index of the lens. And stored in the individual storage unit 312. Further, the evaluation value calculation unit 313 obtains an evaluation value of each individual by an evaluation process determined according to an optimization target, such as calculation of aberration in a lens represented by each individual. While constructing (or reconstructing) the classifier 317, the iterative control unit 314 generates an individual of a new generation by the individual generation unit 311, classifies the individual by the classifier 317, and evaluates by the evaluation value calculation unit 313. The calculation of the value is repeated until the termination condition is satisfied. As described above, the optimizing device realized as the main components of the individual generation unit 311, the individual storage unit 312, the evaluation value calculation unit 313, and the repetition control unit 314 is used for optimizing conditions in various optimization targets. It is possible to

  The configurations in the above embodiment and each modified example may be appropriately combined as long as they do not conflict with each other.

2 Computer 5 Image processing sequence 6, 6a individual 12 Defect detection unit 13 Image storage unit 14 Image processing unit 20 Image processing acquisition device 41 Target image data 43 Target image data 44 Processed image data 51a, 51b Image processing 92 Program 311 Individual generation Unit 312 individual storage unit 313 evaluation value calculation unit 314 repetition control unit 315 classification control unit 316 classifier construction unit 317 classifier 600 element S11 to S20 step

Claims (14)

A computer-based optimization method using a genetic algorithm or genetic programming,
a) storing a plurality of individuals each representing a plurality of conditions to be optimized;
b) obtaining an evaluation value for each of the plurality of individuals by an evaluation process determined according to the optimization target;
c) generating a plurality of new individuals by selection in which the individuals with the higher evaluation values are preferentially included among the plurality of individuals, and a genetic operation including mutation and crossover;
d) obtaining the individual estimated as the optimal solution by repeating the steps b) and c) until the end condition is satisfied;
With
The step d) includes:
d1) Learning is performed using a feature vector obtained from each of a plurality of teacher individuals including the individual generated in step c) and a class based on the evaluation value in each of the plurality of teacher individuals. By constructing a classifier,
d2) By classifying a plurality of new individuals generated in each step c) after the step d1) by the classifier, the evaluation value is obtained in the step b) following the step c). Limiting the required individuals;
An optimization method comprising: The optimization method according to claim 1, wherein
The plurality of individuals respectively represent a plurality of image processing sequences,
In the evaluation processing, a degree of coincidence between a processed image generated by executing an image processing sequence represented by each individual on a predetermined target image and a target image to be generated from the target image is obtained. An optimization method characterized in that: The optimization method according to claim 2, wherein
The individual is an element array in which a plurality of elements are arranged,
The plurality of elements indicate values of a plurality of parameters in a plurality of image processing included in the image processing sequence,
An optimization method, wherein a plurality of feature amounts in the feature vector are values of the plurality of parameters. The optimization method according to claim 2 or 3, wherein
In the evaluation processing, a plurality of processed images generated by executing an image processing sequence represented by each individual on a plurality of target images including the target image, and a plurality of processed images generated from the plurality of target images The degree of coincidence with multiple target images should be calculated,
The classification result by the classifier in the step d2) is a class in which the evaluation value is obtained from the first number of target images in the next step b), and a second number smaller than the first number. An optimization method comprising: a class in which the evaluation value is obtained from a target image; and a class in which the evaluation value is not obtained from a target image. The optimization method according to any one of claims 1 to 4, wherein
In the d2) step and each of the c) steps, generation of a new individual is repeated until the number of individuals classified by the classifier into the class for which the evaluation value is obtained reaches a set individual number. Optimization method to do. The optimization method according to any one of claims 1 to 5,
The optimization method, wherein in the step d), when a predetermined condition is satisfied, the construction of the classifier in the step d1) is performed again. An optimization device using a genetic algorithm or genetic programming,
A storage unit that stores a plurality of individuals that respectively represent a plurality of conditions to be optimized,
An evaluation value calculation unit that obtains an evaluation value for each of the plurality of individuals by an evaluation process determined according to the optimization target,
An individual generation unit that generates a new plurality of individuals by a selection in which the evaluation value is higher among the plurality of individuals, and a genetic operation including mutation and crossover,
By repeating the processing in the evaluation value calculation unit and the individual generation unit until an end condition is satisfied, a repetition control unit that obtains an individual estimated as an optimal solution,
With
The repetition control unit,
Classification by performing learning using a feature vector obtained from each of a plurality of teacher individuals including the individual generated by the individual generation unit and a class based on the evaluation value in each of the plurality of teacher individuals. A classifier construction unit for constructing a vessel;
A classification control unit that classifies a plurality of new individuals generated by the individual generation unit with the classifier, thereby restricting individuals whose evaluation values are obtained in the evaluation value calculation unit,
An optimization device comprising: The optimization device according to claim 7,
The plurality of individuals respectively represent a plurality of image processing sequences,
In the evaluation processing, a degree of coincidence between a processed image generated by executing an image processing sequence represented by each individual on a predetermined target image and a target image to be generated from the target image is obtained. An optimizing device, characterized in that: The optimization device according to claim 8,
The individual is an element array in which a plurality of elements are arranged,
The plurality of elements indicate values of a plurality of parameters in a plurality of image processing included in the image processing sequence,
An optimization apparatus, wherein a plurality of feature amounts in the feature vector are values of the plurality of parameters. The optimization device according to claim 8 or 9, wherein:
In the evaluation processing, a plurality of processed images generated by executing an image processing sequence represented by each individual on a plurality of target images including the target image, and a plurality of processed images generated from the plurality of target images The degree of coincidence with multiple target images should be calculated,
The classification result by the classifier is a class in which the evaluation value is calculated from a first number of target images in the evaluation value calculation unit, and the evaluation value is calculated from a second number of target images smaller than the first number. And a class for which the evaluation value is not obtained. The optimization device according to any one of claims 7 to 10, wherein
After the classifier is constructed, the individual generation unit repeats generation of a new individual until the number of individuals classified by the classifier into the class for which the evaluation value is obtained reaches a set number of individuals. Optimizer to feature. The optimization device according to any one of claims 7 to 11, wherein
The optimization apparatus, wherein when a predetermined condition is satisfied, the repetition control unit causes the classifier constructing unit to construct a classifier again. A program for causing a computer to perform an optimization process using a genetic algorithm or a genetic programming, wherein the execution of the program by the computer causes the computer to:
a) storing a plurality of individuals each representing a plurality of conditions to be optimized;
b) obtaining an evaluation value for each of the plurality of individuals by an evaluation process determined according to the optimization target;
c) generating a plurality of new individuals by selection in which the individuals with the higher evaluation values are preferentially included among the plurality of individuals, and a genetic operation including mutation and crossover;
d) obtaining the individual estimated as the optimal solution by repeating the steps b) and c) until the end condition is satisfied;
And execute
The step d) includes:
d1) Learning is performed using a feature vector obtained from each of a plurality of teacher individuals including the individual generated in step c) and a class based on the evaluation value in each of the plurality of teacher individuals. By constructing a classifier,
d2) By classifying a plurality of new individuals generated in each step c) after the step d1) by the classifier, the evaluation value is obtained in the step b) following the step c). Limiting the required individuals;
A program characterized by comprising: An image processing device,
An image storage unit that stores a captured image of the target object,
An optimization device according to any one of claims 7 to 12, which acquires an image processing sequence;
An image processing unit that executes the image processing sequence on the captured image,
An image processing apparatus comprising:

Images