JP2011214903A - Appearance inspection apparatus, and apparatus, method and program for generating appearance inspection discriminator - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an apparatus for generating an appearance inspection discriminator, fully letting a discriminator which determines whether an inspection object is defectless or not, fully learn, even when there are only a few available images showing a defect on the defective inspection object.SOLUTION: The apparatus for generating the appearance inspection discriminator has: a pass/fail discrimination information acquisition section (15) for acquiring pass/fail discrimination information indicating which an image of the defect in a pseudo-defect image corresponds to, a defectless inspection object, or a defective inspection object, as to a plurality of pseudo-defect images presumably indicating the image of the defect occurring on the surface of the inspection object; a boundary determination section (12) for determining a boundary discriminating the defectless and defective inspection objects based on the pass/fail determination information corresponding to the plurality of pseudo-defect images; a sample generation section (16) for generating a plurality of learning samples which are sets of a characteristic amount concerning the image of the defect and a value indicating a pass/fail determination result of the inspection object with respect to the characteristic amount to be determined according to the boundary; and a discriminator learning section (17) for learning the appearance inspection discriminator, using the plurality of learning samples.

Description

  The present invention relates to an appearance inspection apparatus that determines the quality of an inspection object by analyzing an image obtained by photographing the inspection object, and a generation apparatus for an appearance inspection identifier used in such an appearance inspection apparatus, and The present invention relates to a visual inspection discriminator generation method and a visual inspection discriminator generation computer program.

  Conventionally, in order to save labor in the inspection process or improve inspection accuracy, the inspection image obtained by photographing the inspection object is analyzed to remove defects such as wrinkles on the surface of the inspection object. A technique for detecting and determining whether an inspection object is a non-defective product or a defective product has been studied. In particular, as one of such techniques, for example, a classifier using a machine learning system such as a neural network is trained by using a sample image that is known in advance as a non-defective product. Techniques for determining whether or not an inspection object is a non-defective product using a container have been studied (see, for example, Patent Documents 1 and 2 and Non-Patent Document 1).

For example, in a method called evolutionary image processing disclosed in Non-Patent Document 1, a target image is created by manually painting a portion corresponding to a defect on an inspection image so that it can be distinguished from other portions. . Then, a search is made for a processing flow that can distinguish between a painted portion and an unpainted portion using the target image.
Moreover, the object inspection apparatus disclosed in Patent Document 1 inputs a learning data set to an unsupervised competitive neural network, classifies it by category, and creates a clustering map. Then, when new measurement signal data is input, the apparatus inputs the new data to the competitive neural network after learning, acquires the position of the new data on the map, and acquires a new position from the acquired position. It is determined to which category the data belongs, and the determination result is displayed as a map.
Furthermore, the pseudo defect image creation method disclosed in Patent Document 2 is an image of a portion corresponding to a defect detected in advance in order to increase the number of samples of an image showing a defect that can be used for learning of a classifier. An image sample showing a pseudo defect is created by pasting to other images.

JP 2004-354111 A JP 2006-194657 A

Nagao, "Evolutionary Image Processing", Shosodo, 2002

  However, in general, since there are few inspection objects that are defective products, it is difficult to obtain a large number of inspection images in which defects that cause defects are copied. Therefore, there are cases where a large number of learning samples for defective products that can be used for learning the classifier cannot be prepared. In such a case, in the technique disclosed in Non-Patent Document 1, there are an infinite number of identification boundaries that can be set for a learning sample, so that the discriminator can be sufficiently learned. There was a risk of not being able to. Also, with the technique disclosed in Patent Document 1, since the classifier cannot be sufficiently learned, there is a possibility that the positional relationship between the category and the input data cannot be clarified.

  On the other hand, with the technique disclosed in Patent Document 2, a large number of learning samples representing defects can be prepared in a pseudo manner, so that the classifier can be sufficiently learned. However, there may be many types of defects that cause the inspection object to be determined to be defective. Therefore, a variety of learning samples that cover various defects that should be determined as defective products cannot be created simply by pasting an image showing a specific defect onto another image. In particular, when features that cannot be intuitively grasped by humans, such as texture information such as the spatial frequency or entropy of the image of the defect, are the main factors for determining whether the inspection object is a non-defective product. It was difficult to create an appropriate learning sample. Therefore, even if the technique disclosed in Patent Document 2 is used, there is a possibility that the discriminator cannot be sufficiently learned.

  Therefore, the present invention provides a discriminator that discriminates whether or not the inspection object is a non-defective product based on the inspection image even when only a small number of images showing defects generated in the defective inspection object are obtained. An object of the present invention is to provide a visual inspection discriminator generating apparatus, visual inspection discriminator generating method, and visual inspection discriminator generating computer program that can be learned.

According to the aspect of the first aspect of the present invention, as one aspect of the present invention, an apparatus for generating a visual inspection discriminator is provided. The generation apparatus includes a pseudo defect image generation unit (14) that generates a plurality of pseudo defect images that artificially represent a defect image generated on the surface of the inspection object, and a pseudo defect image for each of the plurality of pseudo defect images. A pass / fail judgment information acquisition unit (15) for acquiring pass / fail judgment information including a pass / fail judgment value indicating whether the image of the upper defect corresponds to a non-defective product of the inspection target or a defective product of the inspection target; A boundary determining unit (12) for determining a boundary for identifying a non-defective product of the inspection object and a defective product of the inspection object from each of the plurality of pseudo defect images and the pass / fail determination information corresponding to each of the pseudo defect images; A sample generation unit (16) that generates a plurality of sets of a feature amount of an image of a defect and a value representing a pass / fail determination result of the inspection object for the feature amount determined according to the boundary as a learning sample; By inputting the feature quantity included in each of the plurality of learning samples into the visual inspection classifier, the visual inspection classifier that identifies whether the inspection object is a non-defective product based on the photographed inspection image And a discriminator learning unit (17) that learns to output a value representing a pass / fail judgment result corresponding to the feature amount.
Even if there are only a small number of learning samples, which are images showing a defect corresponding to a defective product, the visual inspection discriminator generating device identifies whether the inspection object is a non-defective product or a defective product based on the inspection image. The classifier can be sufficiently learned.

According to the second aspect of the present invention, the pseudo defect image generation unit (14) generates a plurality of second pseudo defect images, and the pass / fail judgment information acquisition unit (15) generates a plurality of second pseudo defect images. The pass / fail determination information is acquired for each of the plurality of pseudo defect images and the plurality of second pseudo defect images, and each of the plurality of pseudo defect images and the plurality of second pseudo defect images. It is preferable that the boundary is redetermined from the quality determination information corresponding to each.
As a result, the visual inspection discriminator generating apparatus can more accurately determine a reference boundary for creating a learning sample for learning the visual inspection discriminator.

Furthermore, according to the description of claim 3, the pseudo defect image generation unit (14) includes a pseudo defect image including an image of a defect corresponding to a feature amount within a range in which the quality determination result based on the boundary may be erroneous. Is preferably generated as the second pseudo defect image.
As a result, the visual inspection discriminator generating apparatus can generate a more appropriate pseudo defect image for determining a boundary serving as a reference for creating a learning sample for learning the visual inspection discriminator.

Further, according to claim 4, the boundary determination unit (12) uses the second learning sample which is a set of the feature amount extracted from the pseudo defect image and the pass / fail judgment information corresponding to the pseudo defect image. It is preferable that the boundary is determined based on the second discriminator that outputs the quality determination value of the inspection object by inputting the learned feature amount.
As a result, the visual inspection discriminator generating apparatus can appropriately set even if the boundary serving as a reference for creating the learning sample for learning the visual inspection discriminator is a non-linear boundary.

Moreover, according to the form of Claim 5, the production | generation apparatus of the discriminator for external appearance inspection is provided as another form of this invention. The generation apparatus includes a pseudo defect image generation unit (14) that generates a plurality of pseudo defect images that artificially represent an image of a defect generated on the surface of the inspection object, and a defect extracted from an image obtained by photographing the inspection object. By inputting a feature amount of the image of the image, a plurality of appearance inspection discriminators that output a judgment value indicating whether the image corresponds to a non-defective product of the inspection object or a defective product of the inspection object are used. An identification unit (22) for obtaining a determination value for each of the pseudo defect images, and discrimination between a non-defective product of the inspection object and a defective product of the inspection object determined by the visual inspection classifier among the plurality of pseudo defect images A filter unit (23) that selects a plurality of pseudo defect images corresponding to a determination value close to the boundary, and a defect image on the pseudo defect image is inspected for each of the plurality of pseudo defect images selected by the filter unit (23). Of the object A pass / fail judgment information acquisition unit (15) for acquiring pass / fail judgment information including pass / fail judgment values indicating whether the product corresponds to a product or a defective product to be inspected, and extracted from a plurality of selected pseudo defect images Using a plurality of learning samples, which are a set of feature values and pass / fail judgment values corresponding to the feature amounts, and using an appearance inspection discriminator to identify the feature amounts included in each of the plurality of learning samples for appearance inspection. And a discriminator learning unit (17) for learning to output a pass / fail judgment value corresponding to the feature amount by inputting to the classifier.
Even if there are only a small number of learning samples, which are images showing a defect corresponding to a defective product, the visual inspection discriminator generating device identifies whether the inspection object is a non-defective product or a defective product based on the inspection image. The classifier can be sufficiently learned.

Moreover, according to the form of Claim 6, as another form of this invention, the production | generation method of the discriminator for appearance inspection is provided. In this generation method, a step of generating a plurality of pseudo defect images simulating the defect image generated on the surface of the inspection object, and a defect image on the pseudo defect image is inspected for each of the plurality of pseudo defect images. A step of acquiring pass / fail judgment information including pass / fail judgment values indicating whether the target object corresponds to a non-defective product or an inspected target defective product, each of the plurality of pseudo defect images, and each of the pseudo defect images Determining a boundary for discriminating between a non-defective product to be inspected and a defective product to be inspected from the pass / fail judgment information corresponding to the above, a feature amount for a defect image, and an inspection for the feature amount determined according to the boundary The step of generating a plurality of pairs of values representing the pass / fail judgment result of the target object as learning samples, and identifying whether the test target object is non-defective based on the test image obtained by photographing the test target object The inspection discriminator learns to output a value representing the pass / fail judgment result corresponding to the input feature amount by inputting the feature amount included in each of the plurality of learning samples to the appearance inspection discriminator. Steps.
This method for generating an appearance inspection discriminator discriminates whether the inspection object is a non-defective product or a defective product based on the inspection image even when there are only a few learning samples, which are images of defects corresponding to defective products. The classifier can be sufficiently learned.

Moreover, according to the form of Claim 7, the production | generation method of the discriminator for external appearance inspection is provided as still another form of this invention. In this generation method, a step of generating a plurality of pseudo defect images simulating the defect image generated on the surface of the inspection object and a feature amount of the defect image extracted from the image obtained by photographing the inspection object are input. By using a visual inspection discriminator that outputs a judgment value indicating whether the inspection object corresponds to a non-defective product or a defective inspection object, each of the plurality of pseudo defect images A step of obtaining a judgment value, and a pseudo defect image corresponding to a judgment value close to a discrimination boundary between a non-defective product to be inspected and a defective product to be inspected among the plurality of pseudo defect images, which is determined by the visual inspection classifier For each of the plurality of selected pseudo defect images, whether the defect image on the pseudo defect image corresponds to a non-defective product of the inspection object or a defective product of the inspection object table Using a plurality of learning samples which are a set of a quality determination value including a quality determination value, a feature amount extracted from a plurality of selected pseudo defect images, and a quality determination value corresponding to the feature amount The step of learning the visual inspection discriminator to output the pass / fail judgment value corresponding to the inputted feature amount by inputting the feature amount included in each of the plurality of learning samples to the visual inspection discriminator. Including.
This method for generating an appearance inspection discriminator discriminates whether the inspection object is a non-defective product or a defective product based on the inspection image even when there are only a few learning samples, which are images of defects corresponding to defective products. The classifier can be sufficiently learned.

  According to the aspect of the present invention, a computer program for generating a visual inspection discriminator that causes a computer to generate a visual inspection discriminator is provided as still another aspect of the present invention. This computer program generates a plurality of pseudo defect images that simulate the image of defects generated on the surface of the inspection object, and for each of the plurality of pseudo defect images, the defect image on the pseudo defect image is inspected. A step of acquiring pass / fail judgment information including pass / fail judgment values indicating whether the target object corresponds to a non-defective product or an inspected target defective product, each of the plurality of pseudo defect images, and each of the pseudo defect images Determining a boundary for discriminating between a non-defective product to be inspected and a defective product to be inspected from the pass / fail judgment information corresponding to The step of generating a plurality of pairs of values representing the pass / fail determination result of the target object as learning samples, and whether or not the inspection target object is non-defective based on the inspection image obtained by photographing the inspection target object By inputting the feature quantity included in each of the plurality of learning samples into the appearance inspection classifier, a value representing the pass / fail judgment result corresponding to the input feature quantity is output. And a step of causing the computer to execute the learning step.

  According to a ninth aspect of the present invention, there is provided a computer program for generating a visual inspection discriminator for causing a computer to generate a visual inspection discriminator as still another embodiment of the present invention. This computer program inputs a step of generating a plurality of pseudo defect images simulating the defect image generated on the surface of the inspection object and a feature amount of the defect image extracted from the image obtained by photographing the inspection object. By using a visual inspection discriminator that outputs a judgment value indicating whether the inspection object corresponds to a non-defective product or a defective inspection object, each of the plurality of pseudo defect images A step of obtaining a judgment value, and a pseudo defect image corresponding to a judgment value close to a discrimination boundary between a non-defective product to be inspected and a defective product to be inspected among the plurality of pseudo defect images, which is determined by the visual inspection classifier For each of the plurality of selected pseudo defect images, the defect image on the pseudo defect image corresponds to a non-defective product of the inspection object or A step of obtaining pass / fail judgment information including pass / fail judgment values indicating whether to respond, a feature amount extracted from a plurality of selected pseudo defect images, and a plurality of pass / fail judgment values corresponding to the feature amount Using the learning sample, the visual inspection discriminator outputs the pass / fail judgment value corresponding to the input feature amount by inputting the feature amount included in each of the plurality of learning samples to the visual inspection discriminator. Instructions to cause the computer to execute the learning step.

Moreover, according to the form of Claim 10, an external appearance inspection apparatus is provided as another form of this invention. The appearance inspection apparatus captures an inspection object and generates an inspection image in which the inspection object is captured, and extracts an image of a defect generated on the surface of the inspection object from the inspection image. At least one feature amount in the feature classifier (11) that extracts at least one feature amount of the image of the defect based on the image, and the discriminator learned by the visual inspection discriminator generation device according to any one of the above forms And a pass / fail judgment unit (32) for judging whether or not the inspection object is a non-defective product.
Thereby, this external appearance inspection apparatus can determine correctly the quality of a test target object.

  In addition, the code | symbol in the parenthesis attached | subjected to each said means is an example which shows a corresponding relationship with the specific means as described in embodiment mentioned later.

1 is a schematic configuration diagram of a visual inspection discriminator generation device according to a first embodiment of the present invention. It is a functional block diagram of the process part which the control apparatus by 1st Embodiment has. It is a figure which shows an example of a discriminator. (A)-(c) is a figure which shows an example of a pseudo defect image, respectively. It is a figure which shows an example of the relationship between the feature-value extracted from the pseudo defect image, and the quality determination result. It is a flowchart which shows operation | movement of the discriminator production | generation process for external appearance inspection by 1st Embodiment. It is a functional block diagram of the process part which the control apparatus by 2nd Embodiment has. It is a flowchart which shows operation | movement of the discriminator production | generation process for external appearance inspection by 2nd Embodiment. It is a schematic block diagram of the external appearance inspection apparatus containing the produced | generated visual inspection discriminator.

Hereinafter, a discriminator generating device for appearance inspection apparatus according to a first embodiment of the present invention will be described with reference to the drawings. This discriminator generating device for appearance inspection apparatus generates a pseudo defect image in which defects such as wrinkles on the surface of the inspection object are shown. The visual inspection device discriminator generating device provides the user with information on whether the defect image shown in the pseudo defect image corresponds to a non-defective inspection object or a defective inspection object. The boundary between the non-defective inspection object and the non-defective inspection object for the feature amount of the defect image is roughly determined using the information and the pseudo defect image. Then, the discriminator generating device for the visual inspection apparatus generates a large number of feature values of defect images to be non-defective products and defect image feature values to be defective products as learning samples according to the determined boundary, and the learning samples are generated. The classifier for visual inspection of the inspection object is used to learn.
It should be noted that the inspection object is such that a defect that is a criterion for judging whether or not it is a non-defective product is identifiable from other parts of the inspection object on the inspection object image, for example, the luminance or color of the part corresponding to the defect What is necessary is just to be different from the brightness | luminance or color of another part, for example, it can be set to either glass, a lens, various metal processed materials, resin processed materials, or those which combined them.

FIG. 1 is a diagram illustrating a schematic configuration of a visual inspection device discriminator generation device 1 according to the first embodiment. As illustrated in FIG. 1, the visual inspection device discriminator generation device 1 includes an imaging unit 2, a control device 3, an operation unit 4, and a display unit 5.
The control device 3 includes a communication unit 6, an interface unit 7, a storage unit 8, and a processing unit 9.

The imaging unit 2 includes a camera, for example, and creates a two-dimensional inspection image obtained by capturing the inspection object 100. For this purpose, the imaging unit 2 includes a two-dimensional detector composed of a photoelectric converter such as a CCD or C-MOS sensor, and imaging optics that forms an image of the surface of the inspection object 100 on the two-dimensional detector. Has a system.
The imaging unit 2 outputs the inspection image to the control device 3 every time an inspection image is created.

  The operation unit 4 includes, for example, a keyboard, a pointing device such as a mouse, or a touch sensor. Then, the user interface unit 4 displays a signal representing a determination result on whether the inspection object by the user should be a non-defective product or a non-defective product for the pseudo defect image generated on the display unit 5 that is generated in a pseudo manner. Generated by the operation, and outputs the signal to the control device 3. The details of the pseudo defect image will be described in the description of functions related to the processing unit 9.

The display unit 5 includes, for example, a liquid crystal display or an organic EL display. The display unit 5 displays the inspection image, the pseudo defect image, the quality determination result of the inspection object 100, and the like received from the control device 3. The display unit 5 also provides auxiliary information (for example, a determination result) for allowing the user to input the pass / fail determination result for the pseudo defect image via the operation unit 4 according to the control signal received from the control device 3. Display the operation buttons to be input).
In addition, the operation part 4 and the display part 5 may be comprised integrally, for example like a touchscreen display.

The communication unit 6 includes an interface circuit for connecting the control device 3 to the imaging unit 2. Then, the communication unit 6 passes the inspection image received from the imaging unit 2 to the processing unit 9.
The interface unit 7 includes an interface circuit for connecting the operation unit 4 and the display unit 5 to the control device 3. The interface unit 7 passes the signal received from the operation unit 4 to the processing unit 9. The interface unit 7 outputs a display signal such as an inspection image received from the processing unit 9 to the display unit 5.
The storage unit 8 includes at least one of a semiconductor memory, a magnetic recording medium, its access device, an optical recording medium, and its access device. And the memory | storage part 8 memorize | stores the computer program, various parameters, data, etc. for controlling the discriminator production | generation apparatus 1 for appearance inspection apparatuses.
Further, the storage unit 8 stores the inspection image received from the imaging unit 2 and the pseudo defect image generated by the processing unit 9. Furthermore, the storage unit 8 determines, for each inspection image and each pseudo defect image, whether the defect image shown in the inspection image or the pseudo defect image corresponds to a non-defective inspection object or a defective inspection object. The quality determination information shown is stored.
Further, the storage unit 8 stores various parameters that define the discriminator generated by the processing unit 9.

  The processing unit 9 includes one or a plurality of processors and their peripheral circuits. Then, the processing unit 9 learns a discriminator for identifying whether or not the inspection object 100 is a non-defective product based on the inspection image obtained by photographing the inspection object 100.

FIG. 2 is a functional block diagram of the processing unit 9. The processing unit 9 includes a feature extraction unit 11, a rough boundary determination unit 12, a boundary learning end determination unit 13, a pseudo defect image generation unit 14, and a pass / fail as functional modules implemented by software operating on the processor. The determination information acquisition unit 15, the sample generation unit 16, and the discriminator learning unit 17 are included.
Note that these units included in the processing unit 9 may be configured by independent integrated circuits, firmware, a microprocessor, and the like.
Hereinafter, each part of the processing unit 9 will be described in detail.

The feature extraction unit 11 extracts a feature amount representing the feature of the defect from the inspection image. For this purpose, the feature extraction unit 11 extracts, for example, a defect area in which a defect appears from an inspection image or a pseudo defect image described later. Therefore, for example, a reference image obtained by photographing an inspection object having no defect is stored in the storage unit 8 in advance. Then, the feature extraction unit 11 performs a difference calculation between corresponding pixels between the inspection image or the pseudo defect image and the reference image, and obtains a difference value for each pixel. Then, the feature extraction unit 11 extracts a pixel whose absolute value of the difference value is equal to or greater than a predetermined threshold as a defective area. Note that the predetermined threshold value may be a value at which the cumulative frequency from the minimum difference absolute value reaches a predetermined ratio (for example, 90% of the total number of pixels included in the inspection image).
Note that the feature extraction unit 11 determines the position of the inspection object in the inspection image or the pseudo defect image and the inspection object on the reference image before performing the difference calculation between the inspection image or the pseudo defect image and the reference image. The inspection image or pseudo defect image may be aligned with the reference image so that the positions match. At this time, the feature extraction unit 11 performs, for example, template matching between a template representing a specific part of the inspection object and the inspection image or the pseudo defect image, and a position that best matches the template in the inspection image or the pseudo defect image. By detecting this, the position of the inspection object on the inspection image or the pseudo defect image can be specified.

In addition, the feature extraction unit 11 performs differential filter processing such as a Laplacian filter or a Sobel filter on the entire inspection image or pseudo defect image to create an edge image from which an edge is extracted, and the edge image and the storage unit 8 You may detect the edge of a defect area | region by performing the difference calculation between corresponding pixels between the reference | standard image which is an edge image of the test | inspection object which is the quality goods stored beforehand. In this case, the feature extraction unit 11 sets a region surrounded by the detected edges as a defect region.
Alternatively, the feature extraction unit 11 creates a reduced image obtained by reducing the number of pixels by half in the horizontal direction and the vertical direction of the inspection image, and performs Laplacian filter processing on the reduced image to obtain a differential image. Create And the feature extraction part 11 may extract a defect area | region by performing a binarization process with a predetermined threshold value with respect to a differential image.
Furthermore, the processing unit 9 displays the inspection image or the pseudo defect image on the display unit 5 and allows the user to surround the region corresponding to the defect on the inspection image or the pseudo defect image via the operation unit 4. The defective area may be specified manually.
The defective area is represented by, for example, a binary image in which pixels included in the defective area and pixels included in other areas have different values.

  When the defect area is extracted, the feature extraction unit 11 extracts a feature amount representing the feature of the defect image included in the defect area. For example, values related to the shape of the defect region, such as the area of the defect region, the perimeter of the defect region, the circularity of the defect region, the length of the defect region in the major axis direction or the angle between the major axis direction and the horizontal direction, A value relating to luminance or color in the defect area such as average luminance and average chromaticity, or a value relating to texture such as energy or entropy in the defect area can be used as the feature amount. Alternatively, a feature value such as a histogram of luminance values of pixels in a defect area, a density co-occurrence co-occurrence matrix, or intensity of pixels in a specific frequency band in a spatial frequency image obtained by, for example, two-dimensional Fourier transform of the defect area It is good. The feature extraction unit 11 calculates at least one of these feature amounts. As an example, the feature extraction unit 11 extracts an angle formed by the area of the defect region, the major axis direction of the defect region, and the horizontal direction as a feature amount. Note that the feature extraction unit 11 can set, for example, the number of pixels included in the defect region as the area of the defect region. In addition, the feature extraction unit 11 arbitrarily selects two pixels along the boundary of the defect region in the defect region and determines the distance between the two pixels. It can be determined by obtaining a line connecting the two pixels. Or the feature extraction part 11 is good also considering the major axis direction of the ellipse calculated | required by carrying out ellipse approximation of the defect area as the major axis direction of a defect area | region.

  The feature extraction unit 11 stores the feature amount extracted for the inspection image or the pseudo defect image in the storage unit 8 in association with the corresponding inspection image or the pseudo defect image.

The rough boundary determination unit 12 is an example of a boundary determination unit, and determines a rough boundary that is a boundary for roughly identifying a non-defective product and a defective product of the inspection object 100 based on the feature amount extracted by the feature extraction unit 11. To do. For this purpose, the coarse boundary determination unit 12 extracts the feature amount corresponding to the non-defective inspection target object, the feature amount corresponding to the defective inspection target object extracted from the inspection image or the pseudo defect image, and these features. A coarse boundary model discriminator that determines a coarse boundary is learned by using a set of the result of pass / fail judgment of the inspection object with respect to the quantity as a learning sample.
The coarse boundary model classifier can be, for example, a neural network such as a multilayer perceptron, a support vector machine, or a discriminant function or a Bayesian network.

FIG. 3 is a schematic configuration diagram of an example of a coarse boundary model classifier configured as a multilayer perceptron. The coarse boundary model discriminator 300 has three layers of an input layer 310, an intermediate layer 320, and an output layer 330. Each layer includes one or more neurons, and each neuron 311 in the input layer 310 is connected to each neuron 321 in the intermediate layer 320. Similarly, each neuron 321 in the intermediate layer 320 is connected to the neuron 331 in the output layer 330.
Any of the feature amounts extracted by the feature extraction unit 11 is input to each neuron 311 of the input layer 310. Further, the neuron 331 of the output layer 330 outputs a determination result as to whether or not the inspection object 100 is a non-defective product as a determination value between 0 and 1, for example. This determination value indicates that the closer to 1, the higher the probability that the inspection object 100 is a non-defective product. A determination value of 0.5 indicates that the probability that the inspection object 100 is a non-defective product is equal to the probability that the test object 100 is defective. In other words, the determination value 0.5 corresponds to the rough boundary, and the feature quantity having the determination value 0.5 is located on the rough boundary.

The coarse boundary determination unit 12 learns the coarse boundary model classifier by performing so-called supervised learning using the learning sample. For example, when the coarse boundary model classifier is a multilayer perceptron as described above, the coarse boundary determination unit 12 learns the coarse boundary model classifier by backpropagation.
The coarse boundary determination unit 12 causes the storage unit 8 to store parameters representing the learned coarse boundary model classifier. Note that the parameter representing the learned coarse boundary model classifier is, for example, a weight coefficient for an input from another neuron in each neuron when the coarse boundary model classifier is a multilayer perceptron. If the coarse boundary model discriminator is a support vector machine, the parameter is a support vector that represents the discrimination boundary of the range in which the inspection object is a non-defective item among the feature quantities input to the coarse boundary model discriminator as a learning sample. And a feature amount corresponding to each support vector representing an identification boundary of a range in which the inspection object is a defective product.

The boundary learning end determination unit 13 determines whether or not a rough boundary has been established. Therefore, the boundary learning end determination unit 13 uses a coarse boundary model classifier (hereinafter referred to as a previous coarse boundary model classifier) generated by the previous learning as one method for determining whether or not the coarse boundary is fixed. It is determined whether or not there is a difference between the specified coarse boundary and the coarse boundary defined by the latest coarse boundary model classifier (hereinafter referred to as the latest coarse boundary model classifier). For this purpose, the boundary learning end determination unit 13 determines that each of the previous coarse boundary model classifier and the latest coarse boundary model classifier has all of the pseudo defect images created so far and all of the acquired inspection images, respectively. The feature amount extracted by the feature amount extraction unit 11 is input. If the pass / fail determination result for each inspection image and each pseudo defect image by the latest coarse boundary model classifier is equal to the pass / fail determination result for those images by the previous coarse boundary model classifier, the boundary learning end determination unit 13 Then, it is determined that the rough boundary has been established.
On the other hand, if the pass / fail determination result for any inspection image or pseudo defect image is different between the latest coarse boundary model classifier and the previous coarse boundary model classifier, it is determined that the coarse boundary has not been established.
The boundary learning end determination unit 13 notifies the processing unit 9 of the determination result as to whether or not the rough boundary has been established.

  The pseudo defect image generation unit 14 generates an image including a pseudo defect image as a pseudo defect image in order to create a sample used for learning of the coarse boundary model classifier. For example, the pseudo defect image generation unit 14 photographs a part of the inspection image 100 obtained by photographing the inspection object 100 that is a non-defective product, and the inspection object 100 that is known to be defective in advance. The pseudo defect image is generated by replacing the pixel value included in the region where the defect image cut out from the inspection image obtained in this way is shown. At that time, as disclosed in Japanese Patent Application Laid-Open No. 2006-194657, the pseudo defect image generation unit 14 calculates the luminance difference from the luminance value when there is no luminance unevenness for each pixel included in the luminance unevenness defective portion. Then, processing such as contrast conversion may be performed on the luminance difference, and the processed luminance difference may be added to an inspection image corresponding to a non-defective inspection object.

Alternatively, the pseudo defect image generation unit 14 generates a pseudo defect image by changing the luminance according to the calculation model for a partial region of the inspection image obtained by photographing the inspection object 100 that is a non-defective product. May be. For example, a Gaussian distribution can be used as a calculation model for a spot-like defect such as coloring of a metal object. Further, when a metal object lacks a step, a step function can be used as a calculation model. When the Gaussian distribution is used, the pseudo defect image generation unit 14 generates a pseudo defect image by correcting the luminance of some pixels on the inspection image according to the following equation.
Here, (x ₀ , y ₀ ) represents the horizontal and vertical coordinates of the center position of the pseudo defect, respectively. I (x, y) represents the luminance value of the pixel at the coordinates (x, y) of the inspection image. I ′ (x, y) represents the luminance value of the pixel at the coordinates (x, y) of the pseudo defect image. N (x, y | (x ₀ , y ₀ ), Σ) represents a two-dimensional Gaussian distribution centered on coordinates (x ₀ , y ₀ ). Furthermore, Σ is covariance. The pseudo defect image generation unit 14 can generate a circular defect or a linear defect by changing the covariance value variously. Note that the pseudo defect image generation unit 14 has a Gaussian distribution N (x, y | (x ₀ , y ₀ ), Σ), for example, that has a predetermined threshold value to reduce the amount of calculation required for generating the pseudo defect image. The calculation of the above equation (1) may be performed only for the value of a larger pixel. The predetermined threshold can be set to 1, 5, or 10, for example, when the luminance value of the inspection image is represented by 0 to 255.
In addition, it may be assumed that the brightness of the image of the defect on the inspection image is lower than the brightness of other portions. In this case, the pseudo defect image generation unit 14 adds the luminance obtained according to the Gaussian distribution N (x, y | (x ₀ , y ₀ ), Σ) to the luminance of the pixel of the inspection image in the equation (1). Instead of this, the luminance determined according to the Gaussian distribution N (x, y | (x ₀ , y ₀ ), Σ) may be subtracted from the luminance of the pixels of the inspection image.

  FIG. 4A to FIG. 4C are diagrams each showing an example of a pseudo defect image. In the pseudo defect image 400 shown in FIG. 4A, the defect image 401 is expressed in an elliptical shape. In the pseudo defect image 410 shown in FIG. 4B, the defect image 411 is represented in a substantially circular shape. Further, also in the pseudo defect image 420 shown in FIG. 4C, the defect image 421 is expressed in a substantially circular shape. However, the defect image 421 is smaller than the defect image 411.

  The pseudo defect image generation unit 14 performs a first learning of the rough boundary model classifier by a predetermined number, for example, 10 pseudo defect images in which the size, shape, contrast, color, or the like of the pseudo defects are randomly changed. About 20 sheets are generated and each pseudo defect image is stored in the storage unit 8.

In addition, the pseudo defect image generation unit 14 performs the second and subsequent learning of the coarse boundary model classifier, so that the determination corresponding to the coarse boundary defined by the coarse boundary model classifier generated by the previous learning is performed. A plurality of pseudo defect images are generated based on the feature amount of the defect area corresponding to the determination value within a predetermined range including the value.
In order to specify the feature amount of the defect area corresponding to the determination value within the predetermined range, the pseudo defect image generation unit 14 determines the value within the range of values that can be taken by the feature amount input to the rough boundary identification model, for example. Is changed by a predetermined unit to generate a plurality of feature amounts so as to uniformly cover the entire feature amount space. Alternatively, the pseudo defect image generation unit 14 may generate feature amounts randomly.
The pseudo defect image generation unit 14 inputs the generated feature values to the coarse boundary model classifier to obtain a determination value. Then, the pseudo defect image generation unit 14 selects a feature value whose determination value output by the coarse boundary model classifier is within the predetermined range as a feature value used for learning of the coarse boundary model classifier. The predetermined range is, for example, a range in which the pass / fail judgment result determined according to the coarse boundary may be erroneous. For example, the judgment value output by the coarse boundary model classifier is between 0 and 1, as described above. If it becomes the value of, it can be set as the range equivalent to the determination value of 0.4 or more and 0.6 or less.
Further, if the coarse boundary model classifier is a support vector machine, the pseudo defect image generation unit 14 determines whether the input feature quantity is output by an identification function determined by a kernel function obtained from the support vector. If the absolute value of the difference between the value and the determination value corresponding to the rough boundary (for example, 0) is equal to or smaller than a predetermined threshold value, it may be determined that the determination value for the input feature amount is included in the predetermined range. Note that the predetermined threshold value can be a value obtained by inputting, for example, a feature amount corresponding to a support vector of a non-defective product or a defective product to the discrimination function.

When the pseudo defect image generation unit 14 determines a feature amount corresponding to a determination value within a predetermined range, the pseudo defect image generation unit 14 generates a pseudo defect image corresponding to the feature amount. For example, if the feature amount is the area of the defect region, the pseudo defect image generation unit 14 enlarges or reduces the defect image so that the area of the known defect image becomes the determined value. Alternatively, if the feature amount is an angle formed by the major axis direction of the defect area and the horizontal direction, the pseudo defect image generation unit 14 rotates the image of the known defect so as to coincide with the determined angle.
Alternatively, the pseudo defect image generation unit 14 changes the covariance parameter of the Gaussian distribution in the above equation (1), and determines the feature amount of the defect region generated by the Gaussian distribution. You may match.
The pseudo defect image generation unit 14 stores each generated pseudo defect image in the storage unit 8.

The quality determination information acquisition unit 15 causes the display unit 5 to display the pseudo defect image stored in the storage unit 8. Then, the pass / fail judgment information acquisition unit 15 determines whether the defect image shown in the pseudo defect image displayed on the display unit 5 via the operation unit 4 is a defect that may make the inspection target 100 a non-defective product. Pass / fail judgment information indicating whether the defect 100 should be a defective product or whether it is not a target that does not appear on the inspection image obtained by actually photographing the inspection object 100 is acquired. The pass / fail judgment information includes, for example, pass / fail judgment values corresponding to the judgment values output by the coarse boundary model classifier. For example, when the rough boundary model discriminator outputs a value between 0 and 1 as described above, the pass / fail judgment value corresponding to the defect to be defective is 0, and the pass / fail judgment value corresponding to the defect to be non-defective Is the value of 1.
Similarly, the pass / fail determination information acquisition unit 15 causes the display unit 5 to display an inspection image obtained by photographing the inspection object 100 whether it is unknown whether the product is non-defective, and the defect reflected in the inspection image via the operation unit 4. It may be possible to obtain pass / fail judgment information indicating whether the image is a defect that allows the inspection object 100 to be a non-defective product or a defect that should make the inspection object 100 a defective product.
The pass / fail determination information acquisition unit 15 erases the pseudo defect image obtained from the pass / fail determination information corresponding to the non-target from the storage unit 8. In addition, the quality determination information acquisition unit 15 stores the quality determination information in the storage unit 8 in association with the other pseudo defect images and inspection images.

FIG. 5 is a diagram illustrating an example of the relationship between the feature amount extracted from the pseudo defect image and the pass / fail determination result. In FIG. 5, the horizontal axis represents an angle between the major axis direction of the defect region and the horizontal direction, which is an example of the feature amount, and the vertical axis represents the area of the defect region, which is another example of the feature amount. Each of the plurality of white circles 501 represents a set of feature amounts extracted from one pseudo defect image used last time of the coarse boundary model classifier. A line 502 represents a coarse boundary defined by the coarse boundary model classifier constructed by the previous learning. A plurality of black circles 503 represent feature amounts extracted from a pseudo defect image corresponding to a non-defective inspection object set in accordance with a rough boundary generated for the next learning. A plurality of rhombuses 504 represent feature amounts extracted from a pseudo defect image corresponding to a defective inspection object set in accordance with a rough boundary generated for the next learning.
Here, the quality determination information about the feature quantity 504a for one of the pseudo defect images corresponding to the defective inspection object is corrected to correspond to the non-defective inspection object through the operation of the operation unit 4 by the user. Suppose.
In this case, the rough boundary is corrected so that the feature amount 504a is determined to correspond to a non-defective product to be inspected by the next learning of the rough boundary model classifier. The modified coarse boundary is indicated by line 505.
In this way, the pass / fail determination information of the pseudo defect image corresponding to the feature amount in the vicinity of the coarse boundary is sequentially corrected to correct information by the user. Therefore, by repeating the coarse boundary model classifier learning, the pseudo defect image is used. However, the processing unit 9 can determine a rough boundary that distinguishes a good product and a defective product with a certain degree of accuracy.

  The sample generation unit 16 creates, as a learning sample for the visual inspection classifier, a feature quantity corresponding to a defect that should make the inspection object a non-defective product and a feature quantity that corresponds to a defect that makes the inspection object a defective product. To do. For this purpose, the sample generation unit 16 creates a test feature value set by changing the feature value in a predetermined unit within a range of values that the feature value can take. For example, when the area of the defect region, which is one of the feature quantities, is larger than 0 and a value within an area range of 100 or less, the sample generation unit 16 changes the area by 5 within the area range. Similarly, when the angle with respect to the horizontal direction in the major axis direction of the defect region, which is another feature amount, is a value within an angle range of 0 ° or more and less than 180 °, the sample generation unit 16 sets the angle to the angle. Change by 5 ° within the range. Thereby, the sample generation unit 16 generates a test feature value set including 720 (= 20 × 36) feature values.

  The sample generating unit 16 obtains a determination value for each feature amount by inputting each feature amount included in the generated test feature amount set to the coarse boundary model classifier. Then, the sample generation unit 16 determines from the test feature set that the feature value and the determination value are located on the defective product side of the inspection object 100 with respect to the rough boundary from the rough boundary, and the good value side of the inspection object 100 with respect to the rough boundary. A predetermined number of feature quantities to be positioned are selected, and the selected feature quantities are used as learning samples for a visual inspection classifier. Note that the predetermined number can be a value larger than the number of learning samples used for learning of the coarse boundary model classifier, for example, 100, 500, 1000, or 10000.

It is preferable that the sample generation unit 16 selects the feature amount so that the density of the feature amount selected in the feature amount space becomes uniform as a whole. Or the sample production | generation part 16 may select the feature-value for learning samples so that the number of feature-values may increase, so that it becomes the determination value near a rough boundary.
Further, since there are generally many non-defective inspection objects, it is not difficult to prepare more than the predetermined number of inspection images obtained by actually photographing non-defective inspection objects. Therefore, the sample generation unit 16 may use the feature amount extracted by the feature extraction unit 11 from the inspection image obtained by photographing the inspection target object 100 that is actually a non-defective product as the feature amount related to the defect that should be determined as the non-defective product. Good.
The sample generation unit 16 causes the storage unit 8 to store a set of each selected feature quantity and the quality determination result of the inspection object corresponding to the feature quantity as a learning sample.

The discriminator learning unit 17 learns an appearance inspection discriminator to be used for an appearance inspection of the inspection object 100 using the plurality of learning samples generated by the sample generation unit 16. The visual inspection classifier can also be, for example, a neural network such as a multilayer perceptron, a support vector machine, a discriminant function, or a Bayesian network.
The discriminator learning unit 17 learns the visual inspection classifier according to a supervised learning method according to the configuration of the visual inspection classifier. For example, when the visual inspection classifier is formed of a multilayer perceptron, the classifier learning unit 17 can learn the visual inspection classifier using backpropagation.
Specifically, the discriminator learning unit 17 outputs a determination value corresponding to the pass / fail determination result corresponding to the feature amount when the feature amount is input for each learning sample. Learn classifiers.
Thus, by inputting the feature amount extracted from the inspection image by the feature extraction unit 11, an appearance inspection discriminator that outputs a determination value representing the pass / fail determination result of the inspection object 100 shown in the inspection image. Built.

The discriminator learning unit 17 stores parameters representing the learned visual inspection discriminator in the storage unit 8.
The structure of the visual inspection classifier and the structure of the coarse boundary model classifier may be the same. For example, both the coarse boundary model classifier and the visual inspection classifier may be multilayer perceptrons, or may be both support vector machines.
Alternatively, the visual inspection classifier may have a structure different from that of the coarse boundary model classifier. For example, one of the rough boundary model classifier and the appearance inspection classifier may be a multilayer perceptron, and the other may be a support vector machine. Furthermore, the coarse boundary model classifier and the visual inspection classifier are both multilayer perceptrons. However, in order to reduce the amount of computation required for learning, the number of neurons in the intermediate layer of the coarse boundary model classifier The number may be smaller than the number of neurons in the intermediate layer included in the test discriminator.

FIG. 6 is an operation flowchart of the discriminator generation process in the visual inspection device discriminator generation device 1 according to the first embodiment. This classifier generation process is controlled by the processing unit 9.
The processing unit 9 acquires from the imaging unit 2 at least one inspection image obtained by imaging one or more inspection objects 100 that are known in advance as being non-defective products via the communication unit 6 ( Step S101). Then, the processing unit 9 stores the obtained inspection image in the storage unit 8.
Next, the processing unit 9 determines whether or not a predetermined number or more of inspection images are stored in the storage unit 8 (step S102). For example, the predetermined number is a sufficient number for creating a rough boundary identification model, and is set to about 10 to 20 sheets, for example.

If the predetermined number of inspection images are not stored in the storage unit 8, the pseudo defect image generation unit 14 of the processing unit 9 generates a pseudo defect image (step S103). Then, the pseudo defect image generation unit 14 stores the generated pseudo defect images in the storage unit 8.
The quality determination information acquisition unit 15 of the processing unit 9 displays each pseudo defect image stored in the storage unit 8 on the display unit 5 and acquires quality determination information for the pseudo defect image via the operation unit 4. (Step S104).

In step S102, when a predetermined number of inspection images are stored in the storage unit 8, or after step S104, the feature extraction unit 11 of the processing unit 9 extracts a defect area from the inspection image and the pseudo defect image, A feature amount for the defective area is extracted (step S105). The feature extraction unit 11 stores the extracted feature amount in the storage unit 8 in association with the corresponding inspection image or pseudo defect image.
Next, the rough boundary determination unit 12 of the processing unit 9 includes a plurality of sets of feature amounts of the respective inspection images or pseudo defect images and the pass / fail determination results indicated in the pass / fail determination information corresponding to the feature amounts. A coarse boundary model classifier is learned using the learning sample (step S106). Then, the coarse boundary determination unit 12 stores parameters that define the coarse boundary model classifier in the storage unit 8.

  When the learning of the coarse boundary model classifier is completed, the boundary learning end determination unit 13 of the processing unit 9 uses the coarse boundary defined by the coarse boundary model classifier generated by the previous learning and the latest coarse boundary model classifier. It is determined whether or not there is a difference from the defined rough boundary (step S107). If there is a difference between the coarse boundary between the coarse boundary model classifier generated by the previous learning and the latest coarse boundary model classifier, the boundary learning end determination unit 13 determines that the coarse boundary is not fixed. Then, the pseudo defect image generation unit 14 generates a pseudo defect image based on the feature amount that makes the determination value by the coarse boundary model classifier a value within a predetermined range including the rough boundary (step S108). Thereafter, the processing unit 9 repeats the processes of steps S104 to S107.

On the other hand, if there is no difference between the rough boundary between the rough boundary model classifier generated by the previous learning and the latest rough boundary model classifier, the boundary learning end determination unit 13 determines that the rough boundary has been confirmed. Then, the sample generation unit 16 of the processing unit 9 follows the rough boundary defined by the rough boundary model discriminator, the feature amount of the pseudo defect corresponding to the non-defective inspection object, and the pseudo defect corresponding to the defective inspection object. A predetermined number of feature quantities are generated as learning samples for appearance inspection classifiers (step S109).
Thereafter, the classifier learning unit 17 of the processing unit 9 learns the visual inspection classifier using the visual inspection classifier learning sample (step S110). The discriminator learning unit 17 stores parameters defining the visual inspection discriminator in the storage unit 8.
Then, the processing unit 9 ends the classifier generation process.

  As has been described above, the visual inspection device discriminator generation device according to the first embodiment determines a rough boundary using the pseudo-defective quality determination information of the pseudo defect, and determines a good product based on the rough boundary. The feature amount of the pseudo defect corresponding to the inspection object and the feature quantity of the pseudo defect corresponding to the defective inspection object is generated as a learning sample of the visual inspection classifier. Therefore, even if it is not possible to obtain a large number of inspection images obtained by photographing an actual defective inspection object, the visual inspection device discriminator generation device can generate a large number of appropriate learning samples. And, since such an appropriate learning sample is used to learn a visual inspection discriminator, this visual inspection device discriminator generating device generates a visual inspection discriminator having a high discrimination accuracy between non-defective products and defective products. it can.

Next, a visual inspection device discriminator generation device according to a second embodiment will be described. The visual inspection device discriminator generation device according to the second embodiment directly learns the visual inspection discriminator using the pseudo defect image instead of determining the rough boundary first. At this time, each time the visual inspection discriminator generation device learns the visual inspection discriminator once, the discriminator uses the discriminator to obtain a determination value representing the pass / fail determination result for the pseudo defect image, and the determination value Pass / fail judgment information is acquired from the user via the operation unit for the pseudo defect image included in the predetermined range near the identification boundary. Then, the visual inspection classifier is relearned using a set of pass / fail judgment information corresponding to the pseudo defect image in the vicinity of the identification boundary as a learning sample.
The visual inspection device discriminator generation device according to the second embodiment is different from the visual inspection device discriminator generation device according to the first embodiment only in the function executed by the processing unit of the control device 3. . Therefore, only the processing unit will be described below. For other components of the visual inspection device discriminator generation device according to the second embodiment, refer to the description of the corresponding components of the visual inspection device discriminator generation device according to the first embodiment.

FIG. 7 is a functional block diagram of the processing unit 20 included in the control device 3 according to the second embodiment. The processing unit 20 includes a feature extraction unit 11, a classifier learning unit 17, a learning end determination unit 21, a pseudo defect image generation unit 14, a pass / fail determination information acquisition unit 15, an identification unit 22, and a filter unit 23. Have
In FIG. 7, these units included in the processing unit 20 are denoted by the same reference numerals as the corresponding constituent elements of the processing unit 9 according to the first embodiment shown in FIG. 2. In the following, only those points that are different from the corresponding components of the processing unit 9 according to the first embodiment among these units of the processing unit 20 will be described.

The learning end determination unit 21 applies a feature amount corresponding to a non-defective inspection object and a defective inspection object defined by an appearance inspection identifier (hereinafter referred to as a pre-identifier) generated by the previous learning. It is determined whether there is a difference between the identification boundary with the corresponding feature quantity and the identification boundary defined by the latest appearance inspection classifier (hereinafter referred to as the latest classifier). For this purpose, the learning end determination unit 21 extracts, for each of the previous discriminator and the latest discriminator, the feature amount extraction unit 11 for all of the pseudo defect images created so far and all of the acquired inspection images. Input the feature amount. If the pass / fail judgment result for each inspection image and each pseudo-defect image by the latest discriminator is equal to the pass / fail judgment result for those images by the previous discriminator, the learning end determination unit 21 determines that the visual inspection discriminator is It is determined that it has been sufficiently learned.
On the other hand, if the pass / fail judgment result for any inspection image or pseudo defect image is different between the latest discriminator and the previous discriminator, it is determined that the visual inspection discriminator is not sufficiently learned.
The learning end determination unit 21 notifies the processing unit 20 of the determination result as to whether or not the appearance inspection discriminator has been sufficiently learned.

The identification unit 22 inputs the feature quantity extracted from the pseudo defect image that is not associated with the pass / fail determination information among the pseudo defect images stored in the storage unit 8 to the latest appearance inspection classifier. A determination value representing the quality determination result for the pseudo defect image is obtained. As described with reference to the coarse boundary model classifier in the first embodiment, the visual inspection classifier outputs a determination value between 0 and 1, for example. This determination value indicates that the closer to 1, the higher the probability that the inspection object 100 is a non-defective product. A determination value of 0.5 indicates that the probability that the inspection object 100 is a non-defective product is equal to the probability that the test object 100 is a defective product. In other words, the determination value 0.5 corresponds to the identification boundary, and the feature quantity having the determination value 0.5 is located on the identification boundary. Therefore, in this case, if the determination value is larger than 0.5, the inspection object 100 is determined as a non-defective product, while if the determination value is smaller than 0.5, the inspection object 100 is determined as a defective product.
Then, the identification unit 22 notifies the filter unit 23 of the determination value in association with the corresponding pseudo defect image.

The filter unit 23 selects a pseudo defect image corresponding to a feature amount corresponding to a determination value included in a predetermined range near the identification boundary.
Specifically, the filter unit 23 is the same as the pseudo-defect image generator 14 according to the first embodiment that generates a pseudo-defect image for learning a coarse boundary model classifier for the second and subsequent times. The feature quantity extracted from the image is input to the latest visual inspection classifier, and if the judgment value output by the visual inspection classifier is within a predetermined range, the feature quantity is determined to be in the vicinity of the classification boundary. To do. For example, when the determination value takes a value between 0 and 1, the predetermined range can be 0.4 or more and 0.6 or less.
If the appearance inspection discriminator is a support vector machine, the filter unit 23 discriminates the input feature value from the determination value output by the discrimination function determined by the kernel function obtained from the support vector. If the absolute value of the difference from the determination value (for example, 0) corresponding to the boundary is equal to or smaller than a predetermined threshold value, it may be determined that it falls within the predetermined range. Note that the predetermined threshold value can be a value obtained by inputting, for example, a feature amount corresponding to a support vector of a non-defective product or a defective product to the discrimination function.
The filter unit 23 selects the pseudo defect image determined to have the feature amount located in the vicinity of the identification boundary as the one used for learning by the visual inspection classifier. Then, the filter unit 23 deletes the pseudo defect image that has not been selected from the storage unit 8.

The pass / fail determination information acquisition unit 15 acquires pass / fail determination information about the pseudo defect image selected by the filter 23 via the operation unit 4 and stores the pass / fail determination information in the storage unit 8 in association with the pseudo defect image. . The pass / fail judgment information includes, for example, pass / fail judgment values corresponding to the judgment values output by the visual inspection classifier. For example, when the visual inspection discriminator outputs a value between 0 and 1 as described above, the pass / fail judgment value corresponding to the defect to be defective is 0, and the pass / fail judgment value corresponding to the defect to be non-defective Is the value of 1. The quality determination information about the selected pseudo defect image constitutes a learning sample together with the corresponding pseudo defect image. At this time, the learning sample includes the feature amount extracted from the pseudo defect image selected by the filter unit 23 and the quality determination value included in the quality determination information for the pseudo defect image.
The discriminator learning unit 17 learns an appearance inspection discriminator using the learning sample.

FIG. 8 is an operation flowchart of the discriminator generation process in the visual inspection device discriminator generation device according to the second embodiment. This classifier generation process is controlled by the processing unit 20.
The processing unit 20 acquires from the imaging unit 2 an inspection image obtained by imaging the inspection object 100 that is known in advance as to whether it is a non-defective product via the communication unit 6 (step S201). Then, the processing unit 20 stores the obtained inspection image in the storage unit 8.
Next, the processing unit 20 determines whether or not a predetermined number or more of inspection images are stored in the storage unit 8 (step S202). For example, the predetermined number is a sufficient number for creating a rough boundary identification model, and is set to about 10 to 20 sheets, for example.

If the predetermined number of inspection images are not stored in the storage unit 8, the pseudo defect image generation unit 14 of the processing unit 20 generates a pseudo defect image (step S203). Then, the pseudo defect image generation unit 14 stores the generated pseudo defect images in the storage unit 8.
The quality determination information acquisition unit 15 of the processing unit 20 displays each pseudo defect image stored in the storage unit 8 on the display unit 5, and acquires quality determination information for the pseudo defect image via the operation unit 4. (Step S204).

In step S202, when a predetermined number of inspection images are stored in the storage unit 8, or after step S204, the feature extraction unit 11 of the processing unit 20 extracts a defect region from the inspection image and the pseudo defect image, A feature amount for the defective area is extracted (step S205). The feature extraction unit 11 stores the extracted feature amount in the storage unit 8 in association with the corresponding inspection image or pseudo defect image.
Next, the discriminator learning unit 17 of the processing unit 20 receives the feature amount of each inspection image or pseudo defect image as an input, and uses the learning sample that outputs the pass / fail judgment result indicated in the pass / fail judgment information, A test discriminator is learned (step S206). Then, the discriminator learning unit 17 stores parameters defining the visual inspection discriminator in the storage unit 8.

When the learning of the visual inspection classifier is completed, the learning end determination unit 21 of the processing unit 20 is defined by the identification boundary defined by the visual inspection classifier generated by the previous learning and the latest visual inspection classifier. It is determined whether or not there is a difference with the identified boundary (step S207).
If there is a difference in the discrimination boundary between the visual inspection discriminator generated by the previous learning and the latest visual inspection discriminator, the learning end determination unit 21 determines that learning of the visual inspection discriminator is not sufficient. To do. In this case, the pseudo defect image generation unit 14 generates a plurality of pseudo defect images having various defect images (for example, about several tens to several thousand) (step S208). At this time, it is preferable that the pseudo defect image generation unit 14 generates a pseudo defect image having a defect image different from any of the already generated pseudo defect images. The pseudo defect image generation unit 14 stores the generated pseudo defect image in the storage unit 8.

The feature amount extraction unit 11 extracts the feature amount from the pseudo defect image newly generated in step S208, that is, the pseudo defect image from which the feature amount has not yet been extracted, and the extracted feature amount corresponds to the pseudo defect. The image is stored in the storage unit 8 in association with the image (step S209).
Thereafter, the identification unit 22 of the processing unit 20 inputs a feature amount extracted from the newly generated pseudo defect image to the latest visual inspection classifier, and a determination value representing a quality determination result for the pseudo defect image. Is obtained (step S210). The identification unit 22 notifies the determination value to the filter unit 23 of the processing unit 20. The filter unit 23 selects a pseudo defect image corresponding to the feature amount corresponding to the determination value that is in the vicinity of the identification boundary from the obtained determination value (step S211). Then, the filter unit 23 deletes the pseudo defect image that has not been selected from the storage unit 8.

  Thereafter, the quality determination information acquisition unit 15 displays each pseudo defect image stored in the storage unit 8 on the display unit 5 and acquires quality determination information for the pseudo defect image via the operation unit 4 ( Step S212). Then, the quality determination information acquisition unit 15 stores the quality determination information in the storage unit 8 in association with the corresponding pseudo defect image. Thereafter, the processing unit 20 uses the inspection sample and the pseudo defect image stored in the storage unit 8 as inputs, and uses the learning sample that outputs the pass / fail judgment result indicated in the pass / fail judgment information corresponding to these images. Then, the processing after step S206 is repeated.

On the other hand, in step S207, if there is no difference in the discrimination boundary between the visual inspection discriminator generated by the previous learning and the latest visual inspection discriminator, the learning end determination unit 21 learns the visual inspection discriminator. Is determined to be sufficient. In this case, the processing unit 20 causes the display unit 5 to display a message indicating that learning of the visual inspection classifier has been completed. In addition, the processing unit 20 causes the display unit 5 to display pass / fail determination results for all the pseudo defect images and inspection images generated so far (step S213).
Thereafter, the processing unit 20 ends the classifier generation process.

  As has been described above, the visual inspection device discriminator generation device according to the second embodiment is the one near the identification boundary defined by the visual inspection discriminator that is being learned among the pseudo-defect images generated in a pseudo manner. Pass / fail judgment information is acquired from the user for a pseudo defect image having a feature amount. Therefore, the visual inspection device discriminator generating device can obtain a large number of learning samples effective for learning the visual inspection discriminator. As a result, the discriminator generating device for visual inspection apparatus can identify a visual inspection using a large number of appropriate learning samples even if a large number of inspection images obtained by photographing the actual defective inspection object cannot be obtained. The visual inspection device discriminator generating device can generate a visual inspection discriminator having high discrimination accuracy between a good product and a defective product.

  According to another embodiment, the inspection image used for generating the pseudo defect image may be generated by another device and stored in the storage unit in advance. In this case, the imaging unit may be omitted.

Next, an appearance inspection apparatus using the appearance inspection discriminator generated by any of the appearance inspection discriminator generation apparatuses of the above-described embodiment will be described.
FIG. 9 is a schematic configuration diagram of such an appearance inspection apparatus.
As shown in FIG. 9, the appearance inspection device 30 includes an imaging unit 2, a control device 3, an operation unit 4, and a display unit 5. The control device 3 includes a communication unit 6, an interface unit 7, a storage unit 8, and a processing unit 31. In FIG. 9, these parts of the appearance inspection apparatus 30 have the same reference numerals as the corresponding components of the appearance inspection apparatus discriminator generation apparatus 1 according to the first embodiment shown in FIG. A reference number is attached.
The appearance inspection device 30 is different from the appearance inspection device discriminator generation device 1 according to the first embodiment in the functions realized by the processing unit 31, and the other components of the appearance inspection device 30 are It can be the same as the corresponding component of the discriminator generating device 1 for inspection apparatus. Therefore, for the components other than the processing unit 31 of the appearance inspection device 30, refer to the description of the corresponding components of the appearance inspection device discriminator generation device according to the first embodiment.

The processing unit 31 has one or a plurality of processors and their peripheral circuits. Then, the processing unit 31 determines whether or not the inspection object 100 is a non-defective product based on the inspection image obtained by capturing the inspection object 100 received from the imaging unit 2. For this purpose, the processing unit 31 includes a feature extraction unit 11 and a pass / fail determination unit 32 as functional modules implemented by software operating on the processor.
Note that these units included in the processing unit 31 may be configured by independent integrated circuits, firmware, a microprocessor, and the like.

The feature extraction unit 11 extracts a defect area from the inspection image and extracts a feature amount for the defect area. Note that, when a plurality of defect areas are extracted from the inspection image, the feature extraction unit 11 extracts a feature amount for each defect area. Then, the feature extraction unit 11 passes the extracted feature amount to the pass / fail determination unit 32.
Note that the defect region extraction and feature amount extraction processing by the feature extraction unit 11 are the same as the processing by the feature extraction unit 11 included in the processing unit 9 of the visual inspection device discriminator generation device 1 according to the first embodiment. be able to. Therefore, for details of the processing of the feature extraction unit 11, refer to the description of the feature extraction unit 11 in the first embodiment.

The pass / fail judgment unit 32 reads parameters defining the visual inspection classifier from the storage unit 8 and constructs the visual inspection classifier.
And the quality determination part 32 obtains a quality determination result by inputting the feature-value received from the feature extraction part 11 into the identifier for appearance inspection.
When all the defect areas extracted from the inspection image have obtained a non-defective product determination result, the non-defective product determination unit 32 determines that the inspection object 100 is a good product. On the other hand, when the feature value for any one defective area is input to the visual inspection discriminator, and the pass / fail judgment result that the product is defective is obtained, the pass / fail judgment unit 32 sets the inspection object 100 to be inspected. Judged as defective.
Then, the pass / fail judgment unit 32 notifies the processing unit 31 of the judgment result.

When the processing unit 31 receives the determination result from the pass / fail determination unit 32, the processing unit 31 displays the determination result on the display unit 5. Further, when the processing unit 31 receives a determination result that the inspection target 100 is a defective product, the processing unit 31 stores an inspection image obtained by photographing the inspection target 100 in the storage unit 8.
In addition, the processing unit 31 performs the inspection process of the inspection object 100, for example, the pass / fail judgment result and the inspection image for the inspection object 100 via an interface circuit (not shown) and a communication network connected through the interface circuit. You may output to the system etc. to manage.

  Those skilled in the art can make various modifications in accordance with the embodiment to be implemented within the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Appearance inspection device discriminator generation device 2 Imaging unit 3 Control device 4 Operation unit 5 Display unit 6 Communication unit 7 Interface unit 8 Storage unit 9, 20, 31 Processing unit 11 Feature amount extraction unit 12 Coarse boundary determination unit 13 Boundary learning End determination unit 14 Pseudo defect image generation unit 15 Pass / fail determination information acquisition unit 16 Sample generation unit 17 Discriminator learning unit 21 Learning end determination unit 22 Identification unit 23 Filter unit 32 Pass / fail determination unit 30 Appearance inspection device

Claims (10)

A pseudo-defect image generation unit (14) that generates a plurality of pseudo-defect images that artificially represent an image of a defect generated on the surface of the inspection object;
For each of the plurality of pseudo defect images, a pass / fail judgment value indicating whether a defect image on the pseudo defect image corresponds to a non-defective product of the inspection object or a defective product of the inspection object is included. A pass / fail determination information acquisition unit (15) for acquiring pass / fail determination information;
A boundary determination unit that determines a boundary for identifying a non-defective product of the inspection object and a defective product of the inspection object from each of the plurality of pseudo defect images and the quality determination information corresponding to each of the pseudo defect images. (12)
A sample generation unit (16) for generating a plurality of sets of feature values for the defect image and a value representing a result of quality determination of the inspection object with respect to the feature values determined according to the boundary as a learning sample;
An appearance inspection discriminator for identifying whether or not the inspection object is a non-defective product based on an inspection image obtained by photographing the inspection object, and identifying the feature amount included in each of the plurality of learning samples for the appearance inspection A discriminator learning unit (17) that learns to output a value representing the pass / fail judgment result corresponding to the feature amount by inputting to the classifier;
A discriminator generating device for visual inspection characterized by comprising: The pseudo defect image generation unit (14) generates a plurality of second pseudo defect images,
The quality determination information acquisition unit (15) acquires the quality determination information for each of the plurality of second pseudo defect images,
The boundary determination unit (12) corresponds to each of the plurality of pseudo defect images and the plurality of second pseudo defect images, and each of the plurality of pseudo defect images and the plurality of second pseudo defect images. Re-determining the boundary from the pass / fail judgment information;
The discriminator generator for appearance inspection according to claim 1.   The pseudo defect image generation unit (14) outputs a pseudo defect image including a defect image corresponding to the feature amount within a range in which the quality determination result based on the boundary may be erroneous to the second pseudo image. 3. The visual inspection discriminator generating device according to claim 2, wherein the visual inspection discriminator generating device is generated as a defect image. The boundary determination unit (12) has been learned using a second learning sample that is a set of the feature amount extracted from the pseudo defect image and the quality determination information corresponding to the pseudo defect image. Determining the boundary based on a second discriminator that outputs a pass / fail judgment value of the inspection object by inputting the feature amount;
The discriminator generator for appearance inspection as described in any one of Claims 1-3. A pseudo-defect image generation unit (14) that generates a plurality of pseudo-defect images that artificially represent an image of a defect generated on the surface of the inspection object;
By inputting the feature quantity of the image of the defect extracted from the image obtained by photographing the inspection object, it represents whether it corresponds to a non-defective product of the inspection object or a defective product of the inspection object An identification unit (22) for obtaining the determination value for each of the plurality of pseudo defect images, using a visual inspection classifier that outputs a determination value;
Among the plurality of pseudo defect images, the pseudo defect image corresponding to the determination value close to the identification boundary between the non-defective product of the inspection object and the defective product of the inspection object determined by the visual inspection classifier. A plurality of filter sections (23) for selecting;
For each of the plurality of pseudo defect images selected by the filter unit (23), a defect image on the pseudo defect image corresponds to a non-defective product of the inspection object, or a defective product of the inspection object. A pass / fail determination information acquisition unit (15) for acquiring pass / fail determination information including a pass / fail determination value indicating whether it corresponds;
Using the feature quantity extracted from the selected plurality of pseudo defect images and a plurality of learning samples that are sets of the pass / fail judgment values corresponding to the feature quantity, the visual inspection classifier is A discriminator learning unit (17) for learning to output the pass / fail judgment value corresponding to the feature amount by inputting the feature amount included in each of the plurality of learning samples to the appearance inspection discriminator; ,
A discriminator generating device for visual inspection characterized by comprising: Generating a plurality of pseudo defect images simulating a defect image generated on the surface of the inspection object;
For each of the plurality of pseudo defect images, a pass / fail judgment value indicating whether a defect image on the pseudo defect image corresponds to a non-defective product of the inspection object or a defective product of the inspection object is included. Acquiring pass / fail judgment information;
Determining a boundary for identifying a non-defective product of the inspection object and a defective product of the inspection object from each of the plurality of pseudo defect images and the quality determination information corresponding to each of the pseudo defect images;
Generating as a learning sample a plurality of sets of feature amounts for the defect image and values representing the quality determination results of the inspection object with respect to the feature amounts determined according to the boundaries;
An appearance inspection discriminator for identifying whether or not the inspection object is a non-defective product based on an inspection image obtained by photographing the inspection object, and identifying the feature amount included in each of the plurality of learning samples for the appearance inspection Learning to output a value representing the pass / fail judgment result corresponding to the feature amount by inputting to the device;
A method for generating an identifier for visual inspection, comprising: Generating a plurality of pseudo defect images simulating a defect image generated on the surface of the inspection object;
By inputting the feature quantity of the image of the defect extracted from the image obtained by photographing the inspection object, it represents whether it corresponds to a non-defective product of the inspection object or a defective product of the inspection object Using the visual inspection classifier that outputs a determination value, obtaining the determination value for each of the plurality of pseudo defect images;
Among the plurality of pseudo defect images, the pseudo defect image corresponding to the determination value close to the identification boundary between the non-defective product of the inspection object and the defective product of the inspection object determined by the visual inspection classifier. Multiple selection steps;
For each of the selected plurality of pseudo defect images, whether or not a defect image on the pseudo defect image corresponds to a non-defective product of the inspection object or a non-defective product of the inspection object. Obtaining pass / fail judgment information including a judgment value;
Using the feature quantity extracted from the selected plurality of pseudo defect images and a plurality of learning samples that are sets of the pass / fail judgment values corresponding to the feature quantity, the visual inspection classifier is Learning to output the pass / fail judgment value corresponding to the feature amount by inputting the feature amount included in each of a plurality of learning samples to the visual inspection classifier;
A method for generating an identifier for visual inspection, comprising: Generating a plurality of pseudo defect images simulating a defect image generated on the surface of the inspection object;
For each of the plurality of pseudo defect images, a pass / fail judgment value indicating whether a defect image on the pseudo defect image corresponds to a non-defective product of the inspection object or a defective product of the inspection object is included. Acquiring pass / fail judgment information;
Determining a boundary for identifying a non-defective product of the inspection object and a defective product of the inspection object from each of the plurality of pseudo defect images and the quality determination information corresponding to each of the pseudo defect images;
Generating as a learning sample a plurality of sets of feature amounts for the defect image and values representing the quality determination results of the inspection object with respect to the feature amounts determined according to the boundaries;
An appearance inspection discriminator for identifying whether or not the inspection object is a non-defective product based on an inspection image obtained by photographing the inspection object, and identifying the feature amount included in each of the plurality of learning samples for the appearance inspection Learning to output a value representing the pass / fail judgment result corresponding to the feature amount by inputting to the device;
Is a computer program for generating a visual inspection classifier. Generating a plurality of pseudo defect images simulating a defect image generated on the surface of the inspection object;
By inputting the feature quantity of the image of the defect extracted from the image obtained by photographing the inspection object, it represents whether it corresponds to a non-defective product of the inspection object or a defective product of the inspection object Using the visual inspection classifier that outputs a determination value, obtaining the determination value for each of the plurality of pseudo defect images;
Among the plurality of pseudo defect images, the pseudo defect image corresponding to the determination value close to the identification boundary between the non-defective product of the inspection object and the defective product of the inspection object determined by the visual inspection classifier. Multiple selection steps;
For each of the selected plurality of pseudo defect images, whether or not a defect image on the pseudo defect image corresponds to a non-defective product of the inspection object or a non-defective product of the inspection object. Obtaining pass / fail judgment information including a judgment value;
Using the feature quantity extracted from the selected plurality of pseudo defect images and a plurality of learning samples that are sets of the pass / fail judgment values corresponding to the feature quantity, the visual inspection classifier is Learning to output the pass / fail judgment value corresponding to the feature amount by inputting the feature amount included in each of a plurality of learning samples to the visual inspection classifier;
Is a computer program for generating a visual inspection classifier. An imaging unit (2) for photographing an inspection object and generating an inspection image showing the inspection object;
A feature extraction unit (11) for extracting an image of a defect generated on the surface of the inspection object from the inspection image, and extracting at least one feature amount of the defect based on the image;
A determination is made as to whether or not the inspection object is a non-defective product by inputting the at least one feature amount into the discriminator learned by the visual inspection discriminator generating device according to any one of claims 1 to 5. A pass / fail judgment unit (32) to perform,
An appearance inspection apparatus characterized by comprising: