CN116645314A

CN116645314A - Teacher data generation device, teacher data generation method, and recording medium containing program

Info

Publication number: CN116645314A
Application number: CN202310108322.7A
Authority: CN
Inventors: 盐见顺一
Original assignee: Screen Holdings Co Ltd
Current assignee: Screen Holdings Co Ltd
Priority date: 2022-02-22
Filing date: 2023-01-20
Publication date: 2023-08-25
Also published as: JP2023122140A; KR20230126163A; TW202334900A

Abstract

A teacher data generation device (4) is provided with: an image receiving unit (41) for receiving, from an inspection device (2) that images an object to detect a defect, a defect image of a predetermined size including a detection region of the defect and defect information indicating a range of the detection region in the defect image; a cut-out image generation unit (42) that cuts out, as a cut-out image, an area including the detection area from the defect image based on the defect information; a display control unit (43) for displaying at least a part of the defect image on the display (35); a determination result receiving unit (44) for receiving an operator's input of a determination result of the defect type of the defect image displayed on the display (35); and a teacher data generation unit (45) that generates teacher data by labeling the determination result on the cut image. This makes it possible to easily generate teacher data including an image in which unnecessary areas other than the defective area are reduced.

Description

Teacher data generation device, teacher data generation method, and recording medium containing program

Technical Field

The present invention relates to a technique for generating teacher data.

Background

In an inspection apparatus for detecting a defect by imaging an object, when the defect is detected, a defect image of a predetermined size including a defect region is output. Classification of defect categories represented by such defect images by a learned model (classifier) is considered. In this case, the operator determines (that is, annotates) the defect type of the defect image prepared in advance, generates teacher data obtained by labeling the defect image with one defect type, and learns using a plurality of pieces of teacher data to generate the learned model.

Japanese patent application laid-open No. 2019-87078 (document 1) discloses a method in which an operator inputs a region including a defect in an image, corrects the outer edge of an extended region so that pixels included in the region exceed a predetermined amount, and associates the corrected region with the image to generate learning data.

However, the defect image output from the inspection apparatus is of a fixed size and includes a large number of unnecessary areas other than the defect area. Therefore, even if learning is performed using teacher data including such a defective image, it is difficult to obtain a high-precision learned model. As in the method of document 1, it is also considered that an image with reduced unnecessary areas can be obtained by inputting an area including a defect by an operator, but the operation load becomes large. Therefore, a method of easily generating teacher data including an image in which unnecessary areas other than a defective area are reduced is required.

Disclosure of Invention

The present invention is directed to a teacher data generation device for generating teacher data, and is aimed at easily generating teacher data including an image in which unnecessary areas other than a defective area are reduced.

The teacher data generation device of the present invention comprises: an image receiving unit that receives, from an inspection device that images an object to detect a defect, a defect image of a predetermined size including a detection region of the defect and defect information indicating a range of the detection region in the defect image; a cutout image generation unit that cuts out an area including the detection area from the defect image as a cutout image based on the defect information; a display control unit that displays at least a part of the defect image on a display; a determination result receiving unit that receives an input of a determination result of a defect type by an operator for the defect image displayed on the display; and a teacher data generation unit that generates teacher data by labeling the determination result on the cut image.

According to the present invention, teacher data including an image in which unnecessary areas other than a defective area are reduced can be easily generated.

Preferably, each position of the object belongs to one of a plurality of region types, the defect information includes region type information indicating a region type to which the detection region belongs, the cutout image generation unit stores an expansion amount set for each region type, and the cutout image generation unit includes a region in which the detection region is expanded by the expansion amount determined using the region type information.

Preferably, the object is a printed circuit board, and the plurality of region categories include at least a plating region and a solder resist region.

Preferably, one of a plurality of inspection sensitivities is set for each position of the object, the defect information includes inspection sensitivity information indicating an inspection sensitivity used at the time of detection of the detection region, the cutout image generation unit stores an expansion amount set for each inspection sensitivity, and the cutout image generation unit includes a region in which the detection region is expanded by the expansion amount determined using the inspection sensitivity information.

Preferably, each position of the object belongs to one of a plurality of region types, the defect information includes region type information indicating a region type to which the detection region belongs, and the teacher data generation unit marks the region type to which the detection region belongs on the cut image in addition to the determination result.

The invention also provides a teacher data generation method for generating teacher data. The teacher data generation method comprises the following steps: a) A step of receiving a defect image of a predetermined size including a detection region of a defect and defect information indicating a range of the detection region in the defect image from an inspection apparatus that images an object to detect the defect; b) Cutting out an area including the detection area from the defect image as a cut-out image based on the defect information; c) A step of displaying at least a part of the defect image on a display; d) A step of receiving an input of a determination result of a defect type by an operator for the defect image displayed on the display; and e) labeling the determination result on the cut image to generate teacher data.

Preferably, each position of the object belongs to one of a plurality of region types, the defect information includes region type information indicating a region type to which the detection region belongs, and in the b) step, an expansion amount set for each region type is prepared, and a region in which the detection region is expanded by an expansion amount determined using the region type information is included in the cut image.

Preferably, one of a plurality of inspection sensitivities is set for each position of the object, the defect information includes inspection sensitivity information indicating an inspection sensitivity used for detecting the detection region, and in the b) step, an expansion amount set for each inspection sensitivity is prepared, and a region in which the detection region is expanded by an expansion amount determined using the inspection sensitivity information is included in the cutout image.

Preferably, each position of the object belongs to one of a plurality of region types, the defect information includes region type information indicating a region type to which the detection region belongs, and in the step e), the region type to which the detection region belongs is marked on the cut image in addition to the determination result, and a learned model for defect classification of the one region type is generated using a plurality of teacher data to which the one region type is marked.

The present invention is also directed to a recording medium having recorded thereon a program for causing a computer to generate teacher data. The program of the present invention is executed by a computer, causing the computer to execute: a) A step of receiving a defect image of a predetermined size including a detection region of a defect and defect information indicating a range of the detection region in the defect image from an inspection apparatus that images an object to detect the defect; b) A step of cutting out an area including the detection area from the defect image based on the defect information as a cut-out image; c) A step of displaying at least a part of the defect image on a display; d) A step of receiving an input of a determination result of a defect type by an operator for the defect image displayed on the display; and e) labeling the determination result on the cut image to generate teacher data.

The above objects and other objects, features, aspects and advantages will be clarified by the following detailed description of the present invention with reference to the accompanying drawings.

Drawings

Fig. 1 is a diagram showing a structure of an inspection system.

Fig. 2 is a diagram showing the structure of a computer.

Fig. 3 is a diagram showing the structure of the teacher data generation device.

Fig. 4 is a diagram showing a flow of processing for generating teacher data.

Fig. 5 is a diagram showing an imaged image.

Fig. 6 is a diagram showing an imaged image.

Fig. 7 is a diagram showing an imaged image.

Fig. 8A is a diagram showing a defect image.

Fig. 8B is a diagram showing a defect image.

Fig. 9 is a diagram showing a defect image.

Fig. 10 is a diagram showing the vicinity of the defective region.

Fig. 11 is a diagram showing a defect image.

Fig. 12 is a diagram showing the vicinity of the defective region.

Fig. 13 is a diagram showing the vicinity of the defective region.

Fig. 14 is a diagram showing the vicinity of the defective region.

Fig. 15 is a diagram showing a printed circuit board.

Fig. 16 is a diagram showing a part of the printed circuit board in an enlarged manner.

Fig. 17 is a diagram showing another example of the classifier.

Description of the drawings:

2: inspection apparatus

3: computer with a memory for storing data

4: teacher data generating device

9: printed circuit board with improved heat dissipation

35: display device

41: image receiving section

42: shear image generation unit

43: display control unit

44: determination result receiving unit

45: teacher data generating unit

72: detection area

521. 522: learned model

811: program

S11-S15: step (a)

Detailed Description

(first embodiment)

Fig. 1 is a diagram showing a structure of an inspection system 1 according to a first embodiment of the present invention. The inspection system 1 inspects a printed circuit board as an object. The inspection system 1 includes an inspection device 2 and a computer 3. In fig. 1, the functional structure implemented by the computer 3 is enclosed by a rectangle of broken lines. The inspection apparatus 2 includes an imaging unit, a moving mechanism, and a defect detecting unit, which are not shown. The imaging section images the printed circuit board. The moving mechanism moves the printed circuit board relative to the imaging section. The defect detecting section detects a defect from the image output from the imaging section. When a defect is detected in the defect detection unit, a defect image of a predetermined size (also referred to as a defect block size) including a defect area is output to the computer 3.

Fig. 2 is a diagram showing the structure of the computer 3. The computer 3 has a structure of a general computer system including a CPU31, a ROM32, a RAM33, a fixed disk 34, a display 35, an input unit 36, a reading device 37, a communication unit 38, a GPU39, and a bus 30. The CPU31 performs various arithmetic processing. The GPU39 performs various arithmetic processing related to image processing. The ROM32 stores a basic program. The RAM33 stores various information. The fixed disk 34 stores information. The display 35 displays various information such as an image. The input unit 36 has a keyboard 36a and a mouse 36b for receiving input from an operator. The reading device 37 reads information from a computer-readable recording medium 81 such as an optical disk, a magnetic disk, a magneto-optical disk, or a memory card. The communication unit 38 transmits and receives signals to and from other structures of the inspection system 1 and external devices. The bus 30 is a signal circuit that connects the CPU31, the GPU39, the ROM32, the RAM33, the fixed disk 34, the display 35, the input unit 36, the reading device 37, and the communication unit 38.

In the computer 3, the program 811 is read out from the recording medium 81 as a program product via the reading device 37 in advance and stored in the fixed disk 34. The program 811 may be stored in the fixed disk 34 via a network. The CPU31 and the GPU39 execute arithmetic processing while using the RAM33 and the fixed disk 34 according to the program 811. The CPU31 and the GPU39 function as an arithmetic unit in the computer 3. Other structures may be employed in addition to the CPU31 and the GPU39, which function as an arithmetic unit.

In the inspection system 1, the computer 3 executes arithmetic processing or the like according to the program 811, thereby realizing a functional configuration surrounded by a broken line in fig. 1. That is, the CPU31, GPU39, ROM32, RAM33, fixed disk 34, and their peripheral structures of the computer 3 realize the teacher data generation device 4, learning unit 51, and classifier 52. All or part of these functions may also be implemented by dedicated circuitry. In addition, these functions may be realized by a plurality of computers.

The classifier 52 is a learned model that classifies defects shown in the defect image input from the inspection apparatus 2 as true defects or false defects. The learning unit 51 performs learning using a plurality of teacher data described later, and generates a learned model (classifier 52). The teacher data generation device 4 generates teacher data used in the learning unit 51.

Fig. 3 is a diagram showing the structure of the teacher data generation device 4. The teacher data generating device 4 includes an image receiving section 41, a cut-out image generating section 42, a display control section 43, a determination result receiving section 44, and a teacher data generating section 45. The image receiving unit 41 is connected to the inspection apparatus 2, and receives an input of a defect image or the like from the inspection apparatus 2. The cutout image generation unit 42 cuts out a cutout image to be described later from the defect image. The display control unit 43 is connected to the display 35, and displays a defect image or the like on the display 35. The determination result receiving unit 44 is connected to the input unit 36, and receives an input from an operator via the input unit 36. The teacher data generation section 45 marks the cut image to generate teacher data.

Fig. 4 is a diagram showing a flow of processing of generating teacher data by the teacher data generation device 4. First, the image receiving unit 41 receives a defect image and defect information described later from the inspection apparatus 2 (step S11).

An example of the process of detecting a defect by the inspection apparatus 2 will be described. Fig. 5 is a diagram showing a multi-gradation imaging image obtained by imaging a part of a printed circuit board. For example, the imaged image is a color image. The imaging image may be a gray scale image. Various areas are provided on the main surface of the printed circuit board. Specifically, the plating region plated with a metal such as copper, a solder resist region (hereinafter also referred to as "SR region") provided with a solder resist on the surface, a screen printing region provided with characters, marks, or the like printed on the solder resist, a through-hole region that is an opening of a through-hole, and the like. The SR region may be divided into a first SR region in which the lower layer of the solder resist is a copper foil and a second SR region in which the lower layer of the solder resist is a base material of a printed circuit board, and the colors of the two regions are different. As described above, each position on the main surface of the printed circuit board belongs to any one of a plurality of region categories including a plating region, a first SR region, a second SR region, a screen printing region, and the like.

The example of fig. 5 includes a region 61 representing a plated region and a region 62 representing an SR region, the region 62 including a region 621 representing a first SR region and a region 622 representing a second SR region. In the following description, the regions 61, 62, 621, 622 are also referred to as "plating region 61", "SR region 62", "first SR region 621", and "second SR region 622". For other classes of areas of the printed circuit board, the same names are also used to refer to corresponding areas of the imaged image.

In the defect detection unit of the inspection apparatus 2, for example, the type of region to which each position in the imaged image belongs is specified by referring to design data (CAM data or the like). In each region type, a normal range of gradation values of each color component is set. In the imaging image, the gradation value of each position is compared with the normal range for each color component, and a set of pixels out of the normal range is detected as a defective region. In the example of fig. 5, there is a darker area 71 than the surrounding on the first SR area 621, and this area 71 is a defective area 71 recognized by the operator who observes the imaged image. In fig. 6, the outer edge of the region 72 (hereinafter referred to as "detection region 72") detected as a defect by the inspection apparatus 2 is indicated by a broken line. In the example of fig. 6, the detection region 72 substantially coincides with the defect region 71.

In the inspection apparatus 2, when a defect is detected, an image of a predetermined size including the detection region 72 is acquired as a defect image. Further, defect information indicating the position and shape (including the size) of the detection region 72 in the defect image is acquired. In the detection of the defect, various known methods (inspection logic and the like) may be used, or different methods may be used for each region type.

When the generation of teacher data is started, a plurality of defect images are acquired in advance from a plurality of imaging images for a plurality of printed circuit boards by the inspection device 2. The plurality of defect images are of the same size (defect block size), representing the same-sized areas in the printed circuit board. Further, defect information indicating the position and shape of the detection region 72 is associated with each defect image. In step S11 of fig. 4, a plurality of defect images and defect information of the plurality of defect images are received in the image receiving unit 41. For example, defect information of a plurality of defect images is contained in one list in a state associated with the plurality of defect images, respectively.

Next, the cutout image generation unit 42 cuts out the region including the detection region 72 from each of the defect images as a cutout image (step S12). In the example of fig. 6, as shown in fig. 7, the region of the circumscribed rectangle 73 (indicated by a broken line in fig. 7) of the detection region 72 is cut out as a cut-out image. Each side of the circumscribed rectangle 73 is parallel to the up-down direction (column direction) or the left-right direction (row direction) of the defective image. Depending on the design of the cutout image generation unit 42, a region of the smallest circumscribed rectangle (each side may be inclined with respect to the up-down direction and the left-right direction) that can be set with respect to the detection region 72 may be cut out as a cutout image.

Further, the display control unit 43 displays the defect image on the display 35 (step S13). The image displayed on the display 35 may be all or a portion of the defect image. For example, a cut image of the defective image may be displayed, or an image obtained by enlarging the cut image by a predetermined number of pixels, that is, an image including the detection region 72 and its surroundings may be displayed. In this way, the display control unit 43 displays at least a part of the defect image on the display 35. In one example, thumbnails of a plurality of defective images are displayed in a window arrangement on the display 35, and an operator selects one of the thumbnails of the defective images via the input unit 36, so that at least a part of the defective image (hereinafter referred to as "selected defective image") is displayed on the display 35. The selection of the defect image displayed on the display 35 may be performed by various known methods.

The determination result receiving unit 44 receives an input of a determination result of a true defect or a false defect by the operator for the selection defect image displayed on the display 35 (step S14). In one example, in a window on the display 35, a button representing "true defect" and a button representing "false defect" are provided together with the selection defect image. The operator confirms the selection of the defect image and selects any one of the buttons via the input unit 36, thereby inputting a determination result indicating whether the defect shown in the selection defect image is a true defect or a false defect. The input of the determination result is received by the determination result receiving unit 44. The operator may input the determination result by various known methods.

The teacher data generation unit 45 generates teacher data by labeling the cut image with the determination result (step S15). The teacher data is data including a cut image obtained from a defective image and a determination result made by an operator for the defective image. The teacher data may include a defect image. In practice, the operator inputs the determination results for a plurality of defect images, and generates a plurality of teacher data. Thus, the teacher data generation processing is completed, and a plurality of teacher data (learning data sets) are obtained.

When generating a plurality of pieces of teacher data, the learning unit 51 of fig. 1 performs machine learning so that the output of the classifier for the input of the cutout image in the plurality of pieces of teacher data is substantially the same as the determination result (true defect or false defect) shown in the plurality of pieces of teacher data, thereby generating the classifier. The classifier is a learned model for classifying defects shown in an image into true defects or false defects, and in the generation of the classifier, the values of parameters included in the classifier and the structure of the classifier are determined. Machine learning is performed, for example, by deep learning using a neural network. The machine learning may be performed by a known method other than deep learning. The classifier (actually the value of the parameter, information representing the structure of the classifier) is transmitted and imported into the classifier 52.

When the inspection system 1 inspects the printed circuit board, the inspection device 2 acquires a plurality of imaging images indicating a plurality of positions of the printed circuit board, and inspects whether or not there is a defect in the plurality of imaging images. When a defect is detected, an image of a prescribed size including the detection area 72 is output as a defect image to the classifier 52. In the classifier 52, the defect shown in the defect image is classified as a true defect or a false defect, and the classification result is stored or output to the outside. In the preferred inspection system 1, the region of the circumscribed rectangle 73 of the detection region 72 in the defect image is cut out as a cut-out image in the cut-out image generating section 42 of the computer 3, and the cut-out image is input to the classifier 52, as in the case of generating teacher data. Thus, in the classifier 52, whether the defect shown in the defect image is a true defect or a false defect can be classified with higher accuracy.

Here, a process of a comparative example for generating teacher data will be described. Fig. 8A and 8B show a defect image, and include a defect region 71. In fig. 8A and 8B, parallel oblique lines having a narrower interval than the SR region 62 are applied to the defective region 71, and the defective region 71 substantially coincides with the detection region acquired by the inspection apparatus 2. In addition, in the example of fig. 8B, a set of a plurality of defective portion areas 711 is detected as one defective area 71.

In the process of the first comparative example, the entire defect image is used as an image of teacher data. As shown in fig. 8A and 8B, the defective image generally represents a region that is much larger than the defective region 71, and thus in the first comparative example, the characteristics of an unnecessary region other than the defective region 71 are also used for learning by the learning section 51. In other words, since the image of the teacher data cannot efficiently represent the characteristics of the defective region 71 (detection region), the classification accuracy in the classifier becomes low.

In the processing of the second comparative example, a region of a predetermined size including the defective region 71 is cut out from the defective image, and used as an image of teacher data. In fig. 8A and 8B, a cut-out area A1 cut out from the defect image in the second comparative example is indicated by a two-dot chain line. The size of the sheared area A1 is determined empirically, for example. In the second comparative example, in the image of the teacher data (the image of the cutout area A1), unnecessary areas other than the defective area 71 are reduced as compared with the first comparative example, but are still included to some extent. In addition, as in the example of fig. 8B, when the defective region 71 is relatively large, since the image is exposed from the cutout region A1, the image of the teacher data cannot represent all the features of the defective region 71 (detection region).

In addition, in the first comparative example and the second comparative example, in order to improve the classification accuracy in the classifier, a large amount of teacher data is required, and the number of inputs (the number of comments) of the determination result of the operator for the defective image is increased. Even if a large amount of teacher data is used, a classifier with high accuracy cannot be generated in some cases.

In contrast, in the teacher data generation device 4 of fig. 3, a defect image of a predetermined size including the detection region 72 of the defect and defect information indicating the position and shape of the detection region 72 in the defect image are input from the inspection device 2, and received by the image receiving unit 41. The cut image generating unit 42 cuts out an area including the detection area 72 from the defect image as a cut image based on the defect information. At least a part of the defect image is displayed on the display 35 by the display control unit 43, and the determination result receiving unit 44 receives input of the determination result of the true defect or the false defect by the operator for the displayed defect image. Then, the teacher data generation unit 45 marks the result of the determination on the cut image, and generates teacher data.

This makes it possible to easily generate teacher data including an image (cut image) in which unnecessary areas other than the defective area 71 are reduced. In addition, almost all the features of the defective region 71 are shown in the image. By using teacher data that efficiently represents the characteristics of the defective region 71 in this way, a highly accurate learned model (classifier 52) can be generated with less teacher data, and the number of comments by the operator can be reduced. In fig. 8A and 8B, a circumscribed rectangle 73 of a detection region cut as a cut image is shown by a broken line.

(second embodiment)

Next, a teacher data generation process according to a second embodiment of the present invention will be described. Fig. 9 is a diagram showing a defective image, showing an example in which a defective region 71 exists on the plated region 61. In fig. 9, parallel oblique lines (the same applies to fig. 11 to 14 described later) having a narrower interval than the SR region 62 are marked on the defective region 71. Fig. 10 is an enlarged view showing the vicinity of the defective region 71, and the detection region 72 obtained by the inspection apparatus 2 is darkened (the same applies to fig. 12 to 14 described later). In the case where the defective region 71 is present on the plating region 61, the outer edge of the detection region 72 tends to substantially coincide with the outer edge of the defective region 71 recognized by the operator who observes the defective image, and in fig. 10, the entire detection region 72 substantially overlaps with the entire defective region 71.

Fig. 11 is a diagram showing a defective image, showing an example in which a defective region 71 exists on the SR region 62. Fig. 12 is a diagram showing the vicinity of the defective region 71 in an enlarged manner, and a set of a plurality of detection partial regions 721 is detected as one detection region 72. In fig. 11 and 12, the outer edge of the defective region 71 is indicated by a broken line, and the outer edge of the defective region 71 (i.e., the boundary with the surroundings) is not clear (the same applies to fig. 14 described later). In the case where the defective region 71 exists on the SR region 62, the outer edge of the detection region 72 tends to be smaller than the outer edge of the defective region 71 recognized by the operator who observes the defective image, and in fig. 12, the detection region 72 overlaps only a part of the defective region 71. The defect detection method may be different between the plating region 61 and the SR region 62.

As described above, each position on the main surface of the printed circuit board belongs to one of a plurality of region types, and the inspection device 2 also determines the region type to which each position in the imaged image belongs. In the inspection apparatus 2 of the present example, when a defect is detected, area type information indicating the area type to which the detection area 72 belongs is generated and included in the defect information.

When the teacher data is generated by the teacher data generation device 4, the image receiving unit 41 receives the defect image and the defect information from the inspection device 2 (fig. 4: step S11). As described above, the defect information includes the region category information in addition to the position and shape of the detection region 72 in the defect image. The cutout image generation unit 42 cuts out the region in which the circumscribed rectangle of the detection region 72 extends up, down, left, and right as a cutout image according to the region type to which the detection region 72 belongs (step S12).

Specifically, the number of pixels (natural number: the same applies hereinafter) in which the circumscribed rectangle is expanded up, down, left, and right is set as the expansion amount, and the expansion amount is set in advance for each of the plurality of region types and stored in the cutout image generation unit 42. As described above, since the outer edge of the detection region 72 on the plating region 61 tends to substantially coincide with the outer edge of the defect region 71, the expansion amount for the plating region 61 is a relatively small number of pixels (for example, 0 to 5 pixels). Therefore, in the example of fig. 10 in which the detection region 72 belongs to the plating region 61, as shown in fig. 13, a region of a circumscribed rectangle 73 (indicated by a broken line in fig. 13) of the detection region 72 or a region in which the region is extremely slightly expanded is cut as a cut image. The cut-out image contains almost the entire defect area 71.

Since the outer edge of the detection region 72 on the SR region 62 tends to be smaller than the outer edge of the defect region 71, the expansion amount for the SR region 62 is a relatively large number of pixels (for example, 10 to 20 pixels). Therefore, in the example of fig. 12 in which the detection region 72 belongs to the SR region 62, as shown in fig. 14, the region 74 in which the circumscribed rectangle 73 of the detection region 72 is expanded by the expansion amount is cut as a cut image. In fig. 14, circumscribed rectangle 73 and region 74 are indicated by dotted lines. The cut-out image (i.e., region 74) includes substantially the entire defect region 71. The region 74 in which the circumscribed rectangle 73 of the detection region 72 is expanded by the expansion amount is the same as the circumscribed rectangle of the region in which the detection region 72 is expanded by the expansion amount.

After the teacher data generation device 4 displays the selection defect image on the display 35 (step S13), the operator inputs the determination result of the true defect or the false defect of the selection defect image, and accepts the input (step S14). Then, by labeling the cut image with the determination result, teacher data is generated (step S15). Thereafter, as in the above-described processing example, the classifier 52 is generated using a plurality of pieces of teacher data.

In the inspection of the printed circuit board in the inspection system 1, when a defect is detected in the inspection device 2, an image of a predetermined size including the detection area 72 is output as a defect image to the computer 3, and a classification result of the classifier 52 is obtained. In the preferred examination system 1, as in the case of generating teacher data, the region in which the circumscribed rectangle 73 of the detection region 72 extends up, down, left, and right is clipped as a clipping image according to the region type to which the detection region 72 belongs, and the clipping image is input to the classifier 52. Thus, in the classifier 52, whether the defect shown in the defect image is a true defect or a false defect can be classified with higher accuracy.

As described above, in the present processing example, the area type information indicating the area type to which the detection area 72 belongs is included in the defect information. The cutout image generation unit 42 stores the expansion amounts set for the respective region types, and includes the region in which the detection region 72 is expanded by the expansion amounts specified by the region type information in the cutout image. This can obtain a preferable cut image representing almost the entire defect region 71, and can generate a highly accurate learned model (classifier 52). In the printed circuit board, since the plating region and the SR region account for a large part, the plurality of region types preferably include at least the plating region and the solder resist region from the viewpoint of obtaining a preferable cutout image.

(third embodiment)

Next, a teacher data generation process according to a third embodiment of the present invention will be described. Fig. 15 is a diagram showing the entire printed circuit board 9. In the printed circuit board 9 during manufacture, the removed part contained in the final product is the discarded circuit board region 92. In fig. 15, the discard board region 92 is marked with parallel oblique lines. Fig. 16 is a diagram showing in an enlarged manner a portion B1 surrounded by a broken line in the printed circuit board 9 of fig. 15, and a discard circuit board region 92 is surrounded by a thick broken line. As shown in fig. 16, in the printed circuit board 9, there are small plating areas densely arranged or areas 91 (areas surrounded by thin broken lines in fig. 16) provided with thin wiring patterns.

Since the defects existing in the region 91 have a great influence on the operation of the printed circuit board 9, the inspection device 2 of this processing example sets an inspection sensitivity that is more strict than that of the other regions with respect to the region 91. Hereinafter, the region 91 is referred to as a "first sensitivity setting region 91". On the other hand, since the defect existing in the foregoing discard circuit board region 92 has little influence on the operation of the printed circuit board 9, a loose inspection sensitivity is set with respect to the discard circuit board region 92 as compared with other regions. Hereinafter, the discard circuit board region 92 is referred to as a "second sensitivity setting region 92". In addition, in the region 93 other than the first sensitivity setting region 91 and the second sensitivity setting region 92, an intermediate examination sensitivity is set. Hereinafter, the region 93 is referred to as a "third sensitivity setting region 93".

In this way, any one of a plurality of inspection sensitivities is set at each position of the printed circuit board 9. In the inspection apparatus 2, in the above-described example in which the gradation value of each position of the imaged image is compared with the normal range, the inspection sensitivity is the width of the normal range. The first sensitivity setting region 91 is set with a normal range narrower than the other regions, and the second sensitivity setting region 92 is set with a normal range wider than the other regions. As described above, various methods can be used for detecting defects, and the method for setting the inspection sensitivity can be appropriately changed according to the method for detecting defects.

In the inspection apparatus 2, by referring to, for example, design data (CAM data or the like), it is determined which of the first sensitivity setting region 91, the second sensitivity setting region 92, and the third sensitivity setting region 93 each position in the imaged image belongs to, and a normal range to be compared is obtained. Then, the gray value at the position is compared with the normal range, and a set of pixels out of the normal range is acquired as the detection region 72. In the inspection apparatus 2, the inspection sensitivity information is included in the defect information described above. The inspection sensitivity information is information capable of specifying the inspection sensitivity used when the detection region 72 is detected, and in this processing example, the inspection sensitivity information is information showing any one of the first sensitivity setting region 91, the second sensitivity setting region 92, and the third sensitivity setting region 93.

When the teacher data is generated by the teacher data generation device 4, the image receiving unit 41 receives the defect image and the defect information from the inspection device 2 (fig. 4: step S11). As described above, the defect information includes inspection sensitivity information in addition to the position and shape of the detection region 72 in the defect image. The cutout image generation unit 42 cuts, as a cutout image, an area in which the circumscribed rectangle 73 of the detection area 72 extends up, down, left, and right, according to the inspection sensitivity used in the detection of the detection area 72 (step S12).

Specifically, the number of pixels of the circumscribed rectangle 73 is set as the expansion amount, and the expansion amount is set in advance for each of the plurality of inspection sensitivities, and stored in the cutout image generation unit 42 and prepared. At the most relaxed inspection sensitivity (i.e., in the case where the detection region 72 is located in the second sensitivity setting region 92), the outer edge of the detection region 72 tends to be smaller than the outer edge of the defective region 71, and thus the expansion is set to a relatively large number of pixels α (e.g., 8 to 12 pixels). At the most severe inspection sensitivity (i.e., in the case where the detection region 72 is located in the first sensitivity setting region 91), the outer edge of the detection region 72 tends to substantially coincide with the outer edge of the defective region 71, and thus the expansion is set to a relatively small number of pixels β (e.g., 0 to 3 pixels). At the intermediate inspection sensitivity (i.e., in the case where the detection region 72 is located in the third sensitivity setting region 93), the outer edge of the detection region 72 tends to be slightly smaller than the outer edge of the defect region 71, and therefore the expansion is set to the number of pixels γ (for example, 4 to 7 pixels) between the number of pixels in the case where the inspection sensitivity is most relaxed and the number of pixels in the case where the inspection sensitivity is most severe.

As described above, the maximum expansion amount when the inspection sensitivity is most relaxed and the minimum expansion amount when the inspection sensitivity is most strict. In other words, α > γ > β is satisfied. As a result, the cut image, which is the region in which the circumscribed rectangle 73 of the detection region 72 is expanded by the expansion amount, includes almost the entire defect region 71.

In the inspection of the printed circuit board in the inspection system 1, when a defect is detected in the inspection device 2, an image of a predetermined size including the detection region 72 is output as a defect image to the computer 3, and a classification result of the classifier 52 is obtained. In the preferred examination system 1, similarly to the case of generating teacher data, the region in which the circumscribed rectangle 73 of the detection region 72 extends up, down, left, and right is cut out as a cut-out image according to the examination sensitivity used in the detection of the detection region 72, and the cut-out image is input to the classifier 52. Thus, in the classifier 52, whether the defect shown in the defect image is a true defect or a false defect can be classified with higher accuracy.

As described above, in the present processing example, one of the plurality of inspection sensitivities is set for each position of the printed circuit board, and the defect information includes inspection sensitivity information indicating the inspection sensitivity used in the detection region 72. The shear image generating unit 42 stores the expansion amounts set for the respective inspection sensitivities, and includes the area obtained by expanding the detection area 72 by the expansion amount determined using the inspection sensitivity information in the shear image. This can obtain a preferable cut image representing almost the entire defect region 71, and can generate a highly accurate learned model (classifier 52).

(fourth embodiment)

Next, a teacher data generation process according to a fourth embodiment of the present invention will be described. As described above, each position on the main surface of the printed circuit board belongs to one of the plurality of region categories. When a defect is detected, the inspection apparatus 2 generates area type information indicating the area type to which the detection area 72 belongs, and includes the area type information in the defect information.

In this processing example, steps S11 to S14 in fig. 4 are the same as those in the first embodiment described above. In step S12, as in the second embodiment, the region in which the circumscribed rectangle 73 of the detection region 72 extends up, down, left, and right may be cut out as a cut-out image according to the region type to which the detection region 72 belongs. In addition, as in the third embodiment, the region in which the circumscribed rectangle 73 of the detection region 72 extends up, down, left, and right may be cut as a cut image according to the inspection sensitivity used in detecting the detection region 72.

In the teacher data generation unit 45, the region type to which the detection region 72 belongs is marked on the cut image in addition to the determination result of the true defect or the false defect by the operator for the selected defect image, thereby generating teacher data (step S15). In the teacher data generation processing, a plurality of pieces of teacher data for each region class are generated from a plurality of defect images. Here, a plurality of teacher data for the plating region and a plurality of teacher data for the SR region are generated.

The learning unit 51 performs machine learning using a plurality of pieces of teacher data for the plating region, thereby generating a learned model 521 for the plating region shown in fig. 17. Further, by performing machine learning using a plurality of teacher data for SR regions, a learned model 522 for SR regions is generated.

When the inspection system 1 inspects the printed circuit board, the inspection device 2 acquires a plurality of imaging images indicating a plurality of positions of the printed circuit board, and inspects whether or not there is a defect in the plurality of imaging images. When a defect is detected, a defect image of a prescribed size including the detection area 72 is output to the classifier 52 together with defect information including area type information. In the classifier 52, if the detection region 72 of the defect image belongs to the plating region, the defects shown in the defect image are classified as true defects or false defects using the plating region with the learned model 521. If the detected region 72 of the defect image belongs to the SR region, the SR region is used to classify the defect shown by the defect image as either a true defect or a false defect using the learned model 522.

As described above, in the present processing example, the area type information indicating the area type to which the detection area 72 belongs is included in the defect information. In the teacher data generation unit 45, the region type to which the detection region 72 belongs is marked on the cut image in addition to the determination result of the true defect or the false defect by the operator. Thus, the learning unit 51 can generate a learned model for classifying defects in one region type using a plurality of pieces of teacher data in which the region type is labeled. In this way, by generating a learned model for each region class, classification accuracy can be further improved.

In the teacher data generation device 4 and the teacher data generation method described above, various modifications can be made.

The defect information input from the inspection device 2 to the teacher data generation device 4 is not limited to the position and shape of the detection region 72, as long as the defect information indicates the range of the detection region 72 in the defect image. For example, the defect information may indicate the range of the circumscribed rectangle of the detection area 72 in the defect image (i.e., the range of each of the up-down direction and the left-right direction).

The region of the defect image cut out as the cut-out image may be defined based on the defect information and may include the detection region 72, and is preferably a region substantially circumscribed to the detection region 72. The region substantially circumscribed to the detection region 72 includes not only the region circumscribed to the detection region 72 but also the region circumscribed to the region in which the detection region 72 is expanded by the expansion amount described above.

In the above embodiment, in step S14 of fig. 4, the determination result of the true defect or the false defect with respect to the defect image is input by the operator, but the determination result of the defect type (for example, foreign matter adhesion, film peeling, etc.) other than the true defect and the false defect may be input. That is, the determination result receiving unit 44 receives an input of a determination result (including a determination result of a true defect or a false defect) of the defect type of the defect image displayed on the display 35 by the operator.

In the second embodiment, when the detection region 72 includes portions belonging to two or more different region types, the expansion amount for any one of the two or more region types may be used for the expansion of the detection region 72. From the standpoint of obtaining a preferable cutout image representing almost the entire defective region 71, it is preferable to use the largest expansion amount among the expansion amounts for the two or more region categories.

In the third embodiment, when the detection region 72 includes portions detected with two or more different inspection sensitivities, the expansion of the detection region 72 may use an expansion amount for any one of the two or more inspection sensitivities. From the standpoint of obtaining a preferable cutout image representing almost the entire defect region 71, it is preferable to use the largest expansion amount among the expansion amounts for the two or more inspection sensitivities.

The object to be inspected in the inspection apparatus 2 may be a circuit board such as a semiconductor circuit board or a glass circuit board, in addition to a printed circuit board. Further, the inspection device 2 may detect defects of objects other than the circuit board, such as mechanical components. The teacher data generation means 4 can easily generate the preferable teacher data for generating the learned model for defect classification of various objects.

The structures of the above embodiments and modifications may be appropriately combined as long as they do not contradict each other.

Although the invention has been described and illustrated in detail, the foregoing description is intended to be in all respects illustrative and not restrictive. Accordingly, it can be said that a plurality of modifications and forms can be made without departing from the scope of the present invention.

Claims

1. A teacher data generation device for generating teacher data, comprising:

an image receiving unit that receives, from an inspection device that images an object to detect a defect, a defect image of a predetermined size including a detection region of the defect and defect information indicating a range of the detection region in the defect image;

a cutout image generation unit that cuts out an area including the detection area from the defect image as a cutout image based on the defect information;

A display control unit that displays at least a part of the defect image on a display;

a determination result receiving unit that receives an input of a determination result of a defect type by an operator for the defect image displayed on the display; and

and a teacher data generation unit configured to generate teacher data by labeling the determination result on the cut image.

2. The teacher data generation device according to claim 1, characterized in that,

each position of the object belongs to one of a plurality of region classes,

the defect information includes area category information indicating an area category to which the detection area belongs,

the cutout image generation unit stores the expansion amounts set for the respective region types, and includes the region in which the detection region is expanded by the expansion amounts determined using the region type information in the cutout image.

3. The teacher data generation device according to claim 2, characterized in that,

the object is a printed circuit board, and the plurality of region categories include at least a plating region and a solder resist region.

4. The teacher data generation device according to any one of claims 1 to 3, characterized in that,

One of a plurality of inspection sensitivities is set for each position of the object,

the defect information contains inspection sensitivity information indicating an inspection sensitivity used at the time of detection of the detection area,

the cutout image generation unit stores the expansion amounts set for the respective inspection sensitivities, and includes the detection region expanded region in the cutout image by the expansion amounts determined by using the inspection sensitivity information.

5. The teacher data generation device according to any one of claims 1 to 3, characterized in that,

each position of the object belongs to one of a plurality of region classes,

in addition to the determination result, the teacher data generation section marks the cut image with a region type to which the detection region belongs.

6. A teacher data generation method for generating teacher data, comprising:

a) A step of receiving a defect image of a predetermined size including a detection region of a defect and defect information indicating a range of the detection region in the defect image from an inspection apparatus that images an object to detect the defect;

b) Cutting out an area including the detection area from the defect image as a cut-out image according to the defect information;

c) A step of displaying at least a part of the defect image on a display;

d) Accepting input of a determination result of a defect type by an operator for the defect image displayed on the display; and

e) And a step of generating teacher data by labeling the determination result on the cut image.

7. The teacher data generation method of claim 6, characterized in that,

each position of the object belongs to one of a plurality of region classes,

in the step b), an expansion amount set for each region type is prepared, and a region in which the detection region is expanded by the expansion amount determined using the region type information is included in the cutout image.

8. The teacher data generation method of claim 7, wherein,

9. The teacher data generation method according to any one of claims 6 to 8, characterized in that,

in the step b), an expansion amount set for each inspection sensitivity is prepared, and an area in which the detection area is expanded by the expansion amount determined using the inspection sensitivity information is included in the cutout image.

10. The teacher data generation method according to any one of claims 6 to 8, characterized in that,

each position of the object belongs to one of a plurality of region classes,

in the step e), in addition to the determination result, the cut image is marked with the region type to which the detection region belongs,

a plurality of teacher data labeled with one region class are used to generate a learned model for defect classification of the one region class.

11. A recording medium having recorded thereon a program for causing a computer to generate teacher data, the program being executed by the computer and causing the computer to execute:

b) A step of cutting out an area including the detection area from the defect image as a cut-out image based on the defect information;

c) A step of displaying at least a part of the defect image on a display;

d) A step of receiving an input of a determination result of a defect type by an operator for the defect image displayed on the display; and