JP6902396B2 - Manufacturing equipment with image classification function - Google Patents

Description

The present invention relates to an image classification system for classifying images of articles, a manufacturing apparatus with an image classification function, a classifier for image classification, and an image classification method.

Conventionally, in the inspection of articles (for example, circuit boards, films, etc.), an image classification system has been used in which images obtained by imaging an article to be inspected are classified according to the presence or absence of defects.

For example, the image classification system constructed in the image processing apparatus described in Patent Document 1 has a defect candidate area set based on a defect candidate point detected from an image obtained by capturing an article, and an area excluding the defect candidate area. When a threshold value is set between the non-defect candidate region set for and the image of the article is input, the image of the article is the defect candidate region or the non-defect candidate region based on the threshold value. It is determined which area is included.

Therefore, if the image of the article is included in the defect candidate region, it can be determined that there is a defect, and if the image of the article is included in the non-defect candidate region, it can be determined that there is no defect. It is possible to confirm whether is a good product or a defective product.

JP-A-2015-41164

By the way, in the conventional image classification system, the image is input in order to prevent the image having the defect in the non-defect candidate region from being sorted, that is, to prevent the article having the defect from being judged as a non-defective product. The threshold value was set so that the condition for classifying the image into the non-defective candidate region becomes strict.

As a result, the accuracy of classifying the image into the non-defect candidate region, that is, the accuracy of judging that the article without defects is definitely a non-defective product is improved, but the defect candidate region is also affected by the strict setting of the threshold value. Images of non-defective articles are included, and there is a problem that non-defective products are mistakenly judged as defective products.

Accordingly, the present invention is, above circumstances in view of the, and an object thereof is to provide a picture image classifier with production equipment that Ru can be better yield in image classification.

The image classification system of the present invention
A classification means for classifying the classification image to be classified according to the type of the classification image is provided.
The classification means
A plurality of identified regions for classifying the classification image whose type is identified based on the identification boundary by a plurality of identification boundaries for identifying the type of the classification image, and between the identified regions. It is set so that it is separated from the unidentified area set between them, and it is also set.
When the type of the classification image matches the type indicated by the identification boundary of any of the plurality of identification boundaries, the classification image is classified into the identified area divided into the identification boundaries matching the types. ,
When the type of the classification image does not match the type of any of the plurality of identification boundaries, the classification image is classified into the unidentified region.

According to the image classification system having the above configuration, the classification means classifies the plurality of identified regions and the unidentified regions between the identified regions by a plurality of identification boundaries, and further, types of images for classification. When matches the type indicated by any of the plurality of identification boundaries, the classification image is classified into the identified area divided into the identification boundaries matching the types, and the classification image is displayed. When the type does not match the type of any of the identified areas of the plurality of identification boundaries, the classification image is classified into the unidentified area. Therefore, the classification image whose type can be reliably identified is classified into the identified area. A classification image whose type cannot be reliably determined (that is, a classification image in which the type may be misidentified) is classified into an unidentified area.

Therefore, it is possible to suppress erroneous classification by accurately classifying the classification images classified into the unidentified area by using, for example, a worker or a dedicated device, thereby improving the yield. it can.

In the image classification system of the present invention
The classification means
Based on the learning image according to the type of the classification image, the learning process for setting the plurality of identification boundaries is executed so that the plurality of identified areas and the unidentified area are separated. It may have been done.

According to such a configuration, since the unidentified area is set between the plurality of identified areas as described above, the unidentified area is set in the vicinity of the adjacent areas as compared with the case where the plurality of identified areas are set adjacent to each other. It is possible to reduce the number of required learning images by eliminating the need for a large number of learning images.

Therefore, the image classification system can suppress the burden of the learning process for setting the identification boundary, and can continuously and repeatedly execute the learning process and the image classification process.

In this case, the classification means is
In the learning process
Based on the learning image, a reference classifier that serves as a boundary for distinguishing the types is set.
By setting predetermined threshold values on one side and the other side of the reference classifier with the reference classifier as a reference, the discriminating boundary is configured so as to separate the plurality of identified areas and the unidentified area. It may be configured to do so.

In this way, the unidentified area is set so as to spread with reference to the reference classifier which is the boundary for distinguishing the types, and a plurality of identification areas are set outside the unidentified area. Classification images of unknown type can be aggregated in an unidentified area set between boundaries for identification (identification boundaries).

The manufacturing apparatus with an image classification function of the present invention
It is a manufacturing apparatus with an image classification function that manufactures an article to be inspected by image classification, and is provided with any of the above image classification systems, and is configured to classify and inspect the article by the image classification system.

The classifier for image classification of the present invention is
Multiple identification boundaries are set to identify the type of classification image to be classified,
The plurality of identification boundaries separate a plurality of identified areas for classifying the classification image whose type is identified based on the identification boundary and an unidentified area set between the identified areas. It is configured as follows.

The image classification method of the present invention
A plurality of identified areas set so that the classification image to be classified is divided into a plurality of identification boundaries for identifying the type of the classification image, and the type is based on the identification boundary. When classifying the identified image for classification into one of a plurality of identified areas for classifying the identified images and an unidentified area set between the identified areas.
When the type of the classification image matches the type indicated by any of the plurality of identification boundaries, the classification image is classified into an identified area classified into the identification boundaries matching the types.
When the type of the classification image does not match the type of any of the plurality of identification boundaries, the classification image is classified into the unidentified region.

As described above, according to the images classifier with production equipment of the present invention, an excellent effect that it is possible to improve the yield in image classification.

FIG. 1 is a block diagram of a manufacturing apparatus with an image classification function according to an embodiment of the present invention. FIG. 2 is a block diagram of an inspection device according to the same embodiment. FIG. 3 is a block diagram of an image classification system according to the same embodiment. FIG. 4 is a flowchart of the manufacturing apparatus with an image classification function according to the same embodiment. FIG. 5 is a sub-flow chart of the manufacturing apparatus with an image classification function according to the same embodiment, and is a sub-flow chart of the learning process of the classification means. FIG. 6 is a sub-flow chart of the manufacturing apparatus with an image classification function according to the same embodiment, and is a sub-flow chart of the classification process of the classification means. 7 (a) to 7 (c) are image diagrams of a feature space for explaining the learning process of the classification means according to the embodiment. FIG. 8 is an image diagram of a feature space for explaining a method of constructing a classifier of the classification means according to the embodiment.

Hereinafter, the image classification system according to the embodiment of the present invention will be described with reference to the accompanying drawings.

The image classification system according to the present embodiment is, for example, a system used when inspecting an article such as a substrate or a film, and an image obtained by imaging the inspection object (hereinafter, an image to be classified is used for classification). It is configured to classify according to the presence or absence of defects in the image).

In the image classification system according to the present embodiment, the manufacturing apparatus with an image classification function configured to be able to manufacture and inspect the article by incorporating the image classification system will be described.

As shown in FIG. 1, the manufacturing apparatus 1 with an image classification function includes an inspection device 2 that inspects the article based on an image obtained by imaging the manufactured article, and the inspection device 2 that images the article. It is provided with an image classification system 3 for classifying the obtained images and a post-process processing device 4 for packaging articles after inspection.

As shown in FIG. 2, the inspection device 2 captures an image of an article to be inspected (hereinafter, the article to be inspected is referred to as an inspection object) and an image of the article obtained by the imaging unit. It has an inspection determination means 21 for inspecting an inspection object based on the above.

The image pickup means 20 is configured to take an image of the entire inspection object and acquire an image (inspection image). When there are a plurality of defect candidates (parts where defects may occur) from the inspection image obtained by imaging the entire inspection object, the imaging means 20 has an number of inspection images corresponding to the number of defect candidates. May be configured to obtain more.

The inspection determination means 21 is configured to determine whether or not a defect of the inspection object is reflected in the inspection image (that is, the presence or absence of the defect). Therefore, the inspection determination means 21 determines that the inspection image is either a normal image without defects or a defect image with defects.

In addition, for example, when the inspection object is an article containing a pattern such as a circuit board, the inspection determination means 21 may be used for comparative inspection using a non-defective image as a master or pattern information of the inspection object acquired from CAD data. It suffices if it is configured to extract defects from the inspection image by performing the comparative inspection of.

Further, when the inspection object is a film, the inspection determination means 21 may perform defect enhancement processing so that defects in the inspection image can be found in a binarized state.

Further, in the inspection determination means 21, in order to prevent the defective image from being erroneously determined as a normal image, the determination conditions for determining the inspection image as a normal image are strictly set, and the defect is actually captured. It is preferable to be able to extract all of the existing images.

The image classification system 3 can be constructed in, for example, a device (for example, a personal computer) including a processing device (CPU) for processing information and a storage device for storing information.

As shown in FIG. 3, the image classification system 3 according to the present embodiment includes an image acquisition means 30 for acquiring an image of an inspection object, a classification means 31 for classifying the images acquired by the image acquisition means 30, and the above. According to the type of the image acquired by the image acquisition means 30, the learning input means 32 for inputting the image for learning into the classification means 31 and the classification result of the image by the classification means 31 are output. It has an output means 33.

The image acquisition means 30 is configured to acquire an image acquired by the image pickup means 20 of the inspection device 2, and more specifically, the image determined by the inspection determination means 21 as a defect image is a defect candidate. It is configured to be acquired as an image (an image that may show defects).

The image acquisition means 30 may directly acquire the defect image from the inspection determination means 21, or may acquire the defect image once stored in the storage device. Further, the image acquisition means 30 can reacquire an image from the coordinate information obtained from the inspection determination means by a means capable of capturing a defect candidate more clearly.

The classification means 31 is configured to execute a classification process for classifying the input image and a learning process for constructing a classifier as a reference for the classification in performing the classification process.

The learning input means 32 acquires a group of learning data (hereinafter, referred to as a learning data set) corresponding to the type of the image acquired by the image acquisition means 30, and classifies the learning data set into the classification means 31. Output to.

The learning data is an image obtained by capturing an image of an inspection object or an image created by imitating an image obtained by capturing an image of an inspection object. Further, the type of image is, for example, a place where an image of an inspection object is imaged or a type of image defined for each structure. In the present embodiment, the learning data is referred to as a learning image in the following description.

The learning image may be stored in a storage device built in the terminal in which the image classification system 3 is constructed, or may be stored in an external device connected to the terminal by wire or wirelessly. Good.

The learning data set includes a plurality of learning images (good learning images) that can reliably or substantially reliably determine that the product is non-defective (that is, no defects are shown), and that the data set is defective (that is, defective). It is composed of a plurality of learning images (defective learning images) that can be reliably or substantially reliably determined.

Here, the learning process of the classification means 31 will be described.

In the classification means 31, the learning good product image and the learning defective product image included in the learning data set are sequentially input by the learning input means 32.

Then, the classification means 31 extracts the input feature amount of the learning good product image and the feature amount of the learning defective product image, and generates a classifier from these feature amounts.

Note that FIG. 7A shows images of the feature quantities P1 and P2 output to the feature quantity space V. Further, as shown in FIG. 7B, the reference classifier D divides the feature space V into two regions V1 and V2, so that the input classification image types are two types (this). In the embodiment, it is an image that identifies which type (presence or absence of defect) corresponds to.

In the classification means 31, for example, a classifier generated by using an algorithm such as SVM, AdaBoost, random forest, MT method, and features such as HOG, SHIFT, LBP, Haar-like, and SURF can be used. .. The classification means 31 according to the present embodiment is configured to generate a classifier by SVM, and HOG is adopted as a feature amount.

Subsequently, a threshold value is set on one side from the reference classifier, and another threshold value is set on the other side as well. These two threshold values serve as boundaries for identifying the type of input classification image (two identification boundaries), and the two identification boundaries are used to classify the classification images whose types are identified by the identification boundary. It is set to separate a plurality of identified areas of the above and an unidentified area set between the identified areas.

When the threshold value set on one side, the threshold value set on the other side, the identified area, and the unidentified area are expressed in the feature amount space from the reference classifier, an image as shown in FIG. 7C is obtained. In FIG. 7C, “C1” indicates a normal threshold value, “C2” indicates a defect threshold value, “V1 and V2” indicate an identified region, and “V3” indicates an unidentified region.

In the present embodiment, the normal image is classified into one identified area, and the defective image is classified into the other identified area. Therefore, the threshold value set on one side with reference to the reference classifier is the threshold value for identifying that the classification image is a normal image, and the threshold value set on the other side with reference to the reference classifier is This is a threshold value for identifying that the classification image is a defective image. In the following description, the threshold value for identifying that the classification image is a normal image is referred to as a normal threshold value, and the threshold value for identifying that the classification image is a defective image is referred to as a defect threshold value.

As shown in FIG. 8, the normal threshold value C1 and the defect threshold value C2 set a region in which different types of images are mixed with the reference classifier D in each section divided into the reference classifier D. It is set to include.

As a result, in the classification means 31, the input classification image corresponds to (matches) the type indicated by one discrimination boundary (normal threshold value) (identified as a normal image) and the other. An image that corresponds to (matches) the type indicated by the boundary (defect threshold) (an image that is identified as a defective image) and an image that does not correspond to any type of each identification boundary (an image whose identification result may be incorrect). ) And can be classified.

The output means 33 is configured to output the classification result of the image by the classification means 31 to a display device (for example, a display) for display, or to output to a storage device for storage.

The configuration of the manufacturing apparatus 1 with an image classification function according to the present embodiment is as described above. Subsequently, the operation including the image classification method of the manufacturing apparatus 1 with the image classification function will be described.

As shown in FIG. 4, the manufacturing apparatus 1 with an image classification function inspects the inspection object by the inspection apparatus 2 (S1). The inspection device 2 uses the imaging means 20 to image a plurality of locations of the inspection target, and determines the presence or absence of defects in the plurality of images obtained by the imaging.

Then, the classification image without defects is determined as a normal image, and the classification image with defects is determined as a defect image.

Subsequently, the image acquisition means 30 of the image classification system 3 acquires the defective image as a defect candidate image (S2).

The learning input means 32 acquires a type of learning data set corresponding to the type of the defect candidate image acquired by the image acquisition means 30 (S3), and classifies the learning data set into the classification means 31. (S4).

When the learning good product image and the learning defective product image included in the learning data set are input from the learning input means 32, the classification means 31 executes the learning process (S5).

As shown in FIG. 5, the classification means 31 derives the feature amount of each good product image for learning and each defective product image for learning in the learning process, and outputs the feature amount to the feature amount space (S6).

Then, based on the feature amount output in the feature amount space, a reference classifier that serves as a boundary for distinguishing the types of classification images is set (S7), and the normal threshold value is set on one side and the other side is set on the other side with the reference classifier as a reference. The defect threshold value is set in (S8). In the present embodiment, as described above, the normal threshold value and the defect threshold value serve as identification boundaries for identifying the type of the classification image. Further, in the feature amount space, a plurality of identified regions and an unidentified region set between the identified regions are divided by a normal threshold value and a defect threshold value.

As a result, a classifier for image classification composed of a plurality of (two in the present embodiment) discrimination boundaries is constructed in the feature space.

Subsequently, as shown in FIG. 4, the defect candidate image is classified by the classification means 31 (S9).

In the classification process of the classification means 31, as shown in FIG. 6, the feature amount of the input defect candidate image is extracted (S10), and the score is output by the discriminator trained by the feature amount (S11).

Then, the score is compared with the normal threshold value, and if the score meets the condition set by the normal threshold value (Yes in S12), the defect candidate image is classified into one of the identified regions, that is, the defect candidate image is It is determined (identified) that the image is a normal image (S13).

On the other hand, if the score does not correspond to the condition set by the normal threshold value (No in S12), the score is compared with the defect threshold value, and if the score corresponds to the condition set by the defect threshold value (Yes in S14). ), The defect candidate image is classified into the other identified region, that is, it is determined (identified) that the defect candidate image is a defect image (S15).

Further, when the score does not correspond to the condition set by the defect threshold value (No in S14), the defect candidate image is classified into the unidentified region, that is, the defect candidate image is not classified based on the presence or absence of the defect. It is determined (identified) as an unclassified image (S16).

Then, as shown in FIG. 4, when the defect candidate image is classified by the classification means 31 and then another defect candidate image is not classified (No in S17), the processing of the image classification system 3 is performed. finish.

As described above, according to the image classification system 3 of the manufacturing apparatus 1 with an image classification function according to the present embodiment, a plurality of identification boundaries for identifying the types of images for classification have different types based on the identification boundaries. Since the plurality of identified areas for classifying the identified images for classification and the unidentified area set between the identified areas are set to be separated, the classification means 31 is used. The input classification image corresponds to the type indicated by one identification boundary (normal threshold value) (identified as a normal image) and the type indicated by the other identification boundary (defect threshold value). It can be classified into an image (an image identified as a defective image) and an image that does not correspond to any type of each identification boundary (an image in which the identification result may be incorrect).

Therefore, the image classification system 3 uses a classification image classified as an image whose presence or absence of defects cannot be reliably determined (that is, a classification image classified into an unidentified area), for example, a worker, a dedicated device, or the like. By using and accurately classifying, it is possible to suppress erroneous classification and improve the yield in image classification.

Further, since the classifier in the image classification system 3 sets an unidentified area between the plurality of identified areas and the identified areas, the number of learning images required for learning can be suppressed.

For example, when setting a plurality of identified areas adjacent to each other, it is necessary to accurately construct the boundary between the adjacent identified areas (that is, a classifier capable of accurately classifying the type of the classification image). (To be constructed) is required, so in addition to a plurality of learning images that can be reliably or substantially reliably determined to be non-defective products, and a plurality of learning images that can be reliably or almost reliably determined to be defective products. , It is necessary to input a large amount of learning images having a feature amount appearing near the boundary.

On the other hand, in the image classification system 3 according to the present embodiment, since an unidentified area having a predetermined area is set between the identified areas, the number of learning images required for constructing the classifier is set. Can be suppressed.

Further, the image classification system 3 can reduce the number of learning images required for constructing the classifier, and thus suppresses the burden of executing the learning process for setting the space for classifying the classification images. Therefore, the learning process and the image classification process can be continuously and repeatedly executed online.

The manufacturing apparatus with an image classification function of the present invention is not limited to the above-described embodiment, and it goes without saying that various modifications are made without departing from the gist of the present invention.

In the above embodiment, the case where the image classification system 3 is incorporated into the apparatus for manufacturing the article has been described as an example, but for example, the image classification system 3 can be used independently. In this case, the image classification system 3 may be configured so that the image to be classified and the learning image corresponding to the type of the image to be classified can be input.

In the above embodiment, the classification means 31 has set two identification boundaries (threshold values) for classifying images (classification images), but the present invention is not limited to this configuration. For example, in the classification means 31, three or more identification boundaries (threshold values) for classifying images may be set.

In the above embodiment, the image classification system 3 is configured to classify images for classification according to the presence or absence of defects, but the configuration is not limited to this. For example, the image classification system 3 may be configured to classify the classification image according to the type of defect. To give an example, in the image classification system 3, when the inspection target is a circuit board, whether the defect generated in the circuit board, that is, the defect reflected in the classification image is a pinhole. It may be configured to classify the classification image according to the lack.

In the above embodiment, in the image classification system 3, one reference classifier has been constructed, but the present invention is not limited to this configuration. For example, in the image classification system 3, it is possible to construct two or more independent (discontinuous) reference classifiers.

1 ... Manufacturing device with image classification function, 2 ... Inspection device, 3 ... Image classification system, 4 ... Post-process processing device, 20 ... Imaging means, 21 ... Inspection determination means, 30 ... Image acquisition means, 31 ... Classification means, 32 ... Learning input means, 33 ... Output means, C ... Discriminator, C1 ... Threshold (normal threshold), C2 ... Threshold (defect threshold), ... Reference classifier, P1, P2 ... Feature amount, V ... Feature amount space, V1, V2 ... identified area, V3 ... unidentified area

Claims (2)

A manufacturing device with an image classification function that manufactures articles and inspects the manufactured articles by image classification.
A classification means for classifying the classification image to be classified according to the type of the classification image is provided.
The classification means is used in the above inspection.
Based on the learning image according to the type of the classification image, a plurality of identification boundaries for identifying the type of the classification image are classified, and the classification image whose type is identified based on the identification boundary is classified. A learning process for setting a plurality of identified areas to be separated from each other and an unidentified area set between the identified areas, and a plurality of identification boundaries in which the type of the image for classification is the plurality of identification boundaries. of the and if the number matches the one of the type indicated by the identification boundary, classifies the identification already region image for the classification type is divided into decision boundary that matches, before Symbol classification image types of said plurality An image classification function configured to continuously and continuously execute a classification process for classifying the classification image into the unidentified area when it does not match any of the classification boundary types of the discrimination boundaries. With manufacturing equipment . The classification means
In the learning process, to set the pre-Symbol reference identifier as a boundary distinguishing the types for the basis of the image learning, the predetermined threshold value on one side and the other side of the reference identifier as a reference the reference identifier by setting, image grouping function with manufacturing apparatus according to claim 1 which constitutes the decision boundary so as to partition a plurality of identification finished area and the unidentified region.

Images