TWI775140B - Learning apparatus, inspection apparatus, learning method and inspection method - Google Patents

Abstract

A learning apparatus, an inspection apparatus, a learning method and an inspection method are provided. In the learning apparatus, a second input unit (402) inputs second data obtained by reclassifying temporary defect images in first data according to the presence/absence of a true defect. A learning unit (403) weights the temporary defect images in the second data to generate a learning data set. The learning unit (403) generates a learned model by learning the relationship between a substrate image and the presence/absence of defect by machine learning using the learning data set. In the generation of the learning data set by the learning unit (403), when the temporary defect image is reclassified as having a true defect, the temporary defect image is multiplied by a weighting factor selected according to the degree of importance of the true defect among plurality of weighting factors, and the temporary defect image is classified as “defective”. As a result, it is possible to realize a highly accurate substrate inspection while reducing the capacity of the learning data set.

Description

Learning device, inspection device, learning method, and inspection method

The present invention relates to a technique for inspecting a substrate having a pattern on the surface.

Conventionally, in inspection of a printed wiring board, an image of each region of the printed wiring board is inspected by an inspection apparatus, and an image judged to have a defect (hereinafter, referred to as a "defective image") is selected. Then, a defect confirmation operation (so-called verification operation) is performed on the selected defect image by the operator. The defects selected by the inspection device include true defects with a high possibility of causing problems in quality, and false reports that have substantially no possibility of causing problems in quality. In the defect confirmation operation, the operator uses True defects are visually selected from defect images selected by the inspection device.

On the other hand, in Japanese Patent No. 6512585 (Patent Document 1), it is proposed to create learning by deep learning using a large number of quality image data prepared in advance in an appearance inspection apparatus for parts such as metal bellows binary file, and use the learning binary file to perform image processing on the image to be inspected to determine whether it is good or not.

However, when applying the deep learning technique like Patent Document 1 to the selection of defective images in an inspection apparatus for printed wiring boards, in order to reduce the proportion of false reports included in the defective images and improve inspection accuracy, Then, the number of images required for the learning process becomes large (that is, the capacity of the learning data set becomes large), and there is a concern that a large amount of time is required for learning.

The present invention is a learning device for producing a learning completed model used when inspecting a substrate having a pattern on the surface, and aims to reduce the capacity of the learning data set and realize high-precision inspection of the substrate.

A learning apparatus according to a preferred embodiment of the present invention includes: a first input unit for inputting first data in which a substrate image is classified into a presumed defective image with defects and a presumed good image without defects; a second input a unit for inputting second data in which the assumed defect image in the first data is reclassified by the presence or absence of a true defect; and a learning unit for performing an analysis of the hypothetical defect image in the second data A learning data set is generated by the weighting of the learning data set, and the relationship between the substrate image and the presence or absence of a defect is learned by machine learning using the learning data set, and a learning completion model is produced. In the generation of the learning data set by the learning unit, when the hypothetical defect image is reclassified as having a true defect, among a plurality of weighting coefficients, the corresponding After the weight coefficient selected by the importance of the true defect is multiplied by the hypothetical defect image, the hypothetical defect image is classified as defective, and when the hypothetical defect image is reclassified as not having a true defect , classifying the hypothetical defective image as non-defective.

According to the present invention, it is possible to reduce the capacity of the learning data set and realize high-precision board inspection.

Preferably, the plurality of weighting coefficients become larger as the importance of the region where the true defect is located on the substrate becomes higher.

Preferably, the plurality of weighting coefficients increase as the importance of the type of inspection in which a true defect is detected among the plurality of inspections becomes higher.

The present invention is an inspection apparatus for inspecting a substrate having a pattern on the surface. An inspection apparatus according to a preferred embodiment of the present invention includes an inspection unit that inspects an image to be inspected obtained by photographing a substrate using a learned model created by the above-mentioned learning device.

An inspection apparatus according to another preferred embodiment of the present invention includes: the above-mentioned learning device; and an inspection unit that inspects an image to be inspected obtained by photographing a substrate using a learned completed model created by the learning device.

Preferably, the second input unit of the learning device inputs a new image that is reclassified by the presence or absence of a true defect, using the defect image obtained by the inspection unit as the hypothetical defect image. the second data, the learning section performs weighting on the assumed defect image in the new second data to generate a new learning data set, and a machine using the new learning data set is used. Learning, for the learning completion model, the relationship between the substrate image and the presence or absence of defects is relearned.

Preferably, it further includes a photographing unit for photographing the substrate to obtain the image to be inspected.

The present invention is a learning method for learning a completed model used when inspecting a substrate having a pattern on the surface. A learning method according to a preferred embodiment of the present invention includes: a) a step of inputting first data in which a substrate image is classified into a presumed defective image with defects and a presumed good image without defects; b) inputting a borrowed image the step of reclassifying said putative defect images in said first data by the presence or absence of true defects in a second profile; and c) weighting said putative defect images in said second profile to obtain The steps of generating a learning data set, learning the relationship between the substrate image and the presence or absence of defects by machine learning using the learning data set, and creating a learning completion model, the learning data in the step c) In the generation of the set, when the hypothetical defect image is reclassified as having a true defect, among a plurality of weighting coefficients, the weight coefficient selected corresponding to the importance of the true defect and the hypothetical defect image After multiplication, the putative defect image is classified as defective, and when the putative defect image is reclassified as not having a true defect, the putative defect image is classified as non-defective.

Preferably, the plurality of weighting coefficients become larger as the importance of the region where the true defect is located on the substrate becomes higher.

Preferably, the plurality of weighting coefficients increase as the importance of the type of inspection in which a true defect is detected among the plurality of inspections becomes higher.

The present invention is an inspection method for inspecting a substrate having a pattern on the surface. An inspection method according to a preferred embodiment of the present invention includes: utilizing the The learning completion model created by the learning method is a step of inspecting the image to be inspected obtained by photographing the substrate.

The inspection method of another preferred embodiment of the present invention includes: d) the step of making a learning completion model by the learning method described in any one of the technical solutions 8 to 10; d) A step of inspecting the image to be inspected obtained by photographing the substrate in the learning completion model produced in the step.

Preferably, it further comprises: f) inputting the defect image obtained in the step e) as the hypothetical defect image, and reclassifying new second data based on the presence or absence of true defects; step; and g) performing weighting on the assumed defect image in the new second data to generate a new learning data set, by using the machine learning of the new learning data set, for all Describe the steps of learning the completed model and then learning the relationship between the substrate image and the presence or absence of defects.

Preferably, it further includes a step of photographing the substrate to obtain the inspected image.

The above object and other objects, features, aspects and advantages will be further clarified by the detailed description of the present invention below with reference to the accompanying drawings.

1: Inspection device

2: Device body

3: The first computer

4: Second computer

9: Substrate

21: Filming Department

22: Platform

23: Platform moving mechanism

30, 40: busbar

31, 41: CPU

32, 42: ROM

33, 43: RAM

34, 44: Fixed disk

35, 45: Display

36, 46: Input part

36a, 46a: Keyboard

36b, 46b: Mouse

37, 47: Reader

38, 48: Communications Department

39, 49: GPU

81, 82: Recording media

93a: Small area plated area

93b: Medium area plated area

93c: Large area plated area

94a: Thin line SR region

94b: Standard wiring SR area

94c: Substrate SR region

811, 821: Program

211: Lighting Department

212: Optical System

213: Shooting Elements

301, 404: Storage Department

302: Inspection Department

303: Control Department

401: The first input part

402: Second input part

403: Learning Department

S11~S15, S21~S24, S31~S33: Steps

FIG. 1 is a diagram showing a configuration of an inspection apparatus.

FIG. 2 is a diagram showing a configuration of a first computer.

FIG. 3 is a diagram showing a functional configuration realized by a first computer.

FIG. 4 is a diagram showing a configuration of a second computer.

FIG. 5 is a diagram showing a functional configuration realized by a second computer.

FIG. 6 is a diagram showing a flow of creating a learning completion model.

FIG. 7 is a diagram showing a state in which a learning completed model is created.

FIG. 8 is a diagram showing various regions on the substrate.

FIG. 9 is a diagram showing various regions on the substrate.

FIG. 10 is a diagram showing a flow of inspection of a substrate.

FIG. 11 is a diagram showing a flow of re-learning of a learned-completed model.

FIG. 1 is a diagram showing the configuration of an inspection apparatus 1 according to an embodiment of the present invention. The inspection apparatus 1 is, for example, an apparatus for inspecting the appearance of the printed wiring board 9 (hereinafter, also simply referred to as "the board 9") before the electronic components are mounted. A pattern (for example, a wiring pattern or an electrode pattern formed of copper) is formed on the surface of the substrate 9 . On the substrate 9, there are, for example, a region where the copper plating layer is exposed, a region where the copper wiring is covered with a solder resist serving as a protective film, and a region where the solder resist is directly arranged on the surface of the base material.

The inspection apparatus 1 includes an apparatus body 2 for imaging a substrate 9 , a first computer 3 , and a second computer 4 . Each of the first computer 3 and the second computer 4 is a processing device including an arithmetic unit. The first computer 3 also controls the overall operation of the inspection apparatus 1 . The second computer 4 is a learning device that creates a learning completed model to be described later. The apparatus main body 2 includes an imaging unit 21 , a stage 22 holding the substrate 9 , and a stage moving mechanism 23 . The imaging unit 21 images the substrate 9 to acquire an image. The image is, for example, a multi-grayscale color image. The platform moving mechanism 23 relatively moves the platform 22 with respect to the imaging unit 21 verb: move.

The imaging unit 21 includes an illumination unit 211 that emits illumination light, an optical system 212 , and an imaging element 213 . The optical system 212 guides illumination light toward the substrate 9 and guides light from the substrate 9 toward the imaging element 213 . The imaging element 213 converts the image of the substrate 9 formed by the optical system 212 into an electrical signal. The lighting unit 211 includes lamps such as light emitting diodes (LEDs) and light bulbs, and optical components such as lenses and reflecting members that adjust light from the lamps. The optical system 212 includes a plurality of optical components such as lenses and half mirrors. The imaging element 213 is, for example, a two-dimensional image sensor. The imaging element 213 may be a one-dimensional image sensor, and in this case, the imaging is performed while moving the stage 22 .

The stage moving mechanism 23 includes a ball screw, a guide rail, a motor, and the like. Of course, as the stage moving mechanism 23, various mechanisms can be used, for example, a linear motor can be used. The first computer 3 controls the platform moving mechanism 23 and the photographing unit 21 , whereby the platform 22 moves in the horizontal direction, and a desired area on the substrate 9 is photographed. The stage 22 is a holding portion that holds the substrate 9 . The holding of the substrate 9 can be performed by various methods. For example, a groove is formed on the stage 22 and suction is performed from a suction port formed in the groove, whereby the substrate 9 is attracted to the stage 22 . Many suction ports can also be formed on the platform 22 to suck the substrate 9 . The platform 22 may also be formed of a porous material, and suction may be performed from the porous material. The platform 22 can also hold the substrate 9 by a mechanical mechanism.

FIG. 2 is a diagram showing the configuration of the first computer 3 . The first computer 3 has the structure of a general computer system, and the general computer system includes: a central processing unit (Central Processing Unit, CPU) 31, a read-only memory (Read Only Memory) Memory, ROM) 32, random access memory (Random Access Memory, RAM) 33, fixed disk 34, display 35, input unit 36, reading device 37, communication unit 38, Graphics Processing Unit (GPU) ) 39, and the bus bar 30. The CPU 31 performs various arithmetic processing. The GPU 39 performs various arithmetic processing related to image processing. The ROM 32 stores basic programs. The RAM 33 stores various information. The fixed disk 34 stores information. The display 35 displays various information such as images. The input unit 36 includes a keyboard 36a and a mouse 36b that receive input from the operator. The reading device 37 reads information from a computer-readable recording medium 81 such as an optical disk, a magnetic disk, a magneto-optical disk, and a memory card. The display 35, the keyboard 36a, the mouse 36b, and the reading device 37 are connected to the bus bar 30 via the interface I/F. The communication unit 38 transmits and receives signals to and from other components of the inspection apparatus 1 and external devices. The bus bar 30 is a signal circuit connecting the CPU 31 , the GPU 39 , the ROM 32 , the RAM 33 , the fixed disk 34 , the display 35 , the input unit 36 , the reading device 37 , and the communication unit 38 .

In the first computer 3 , the program 811 is read out from the recording medium 81 via the reading device 37 in advance and stored in the fixed disk 34 . The program 811 can also be stored in the fixed disk 34 via the network. The CPU 31 and the GPU 39 execute arithmetic processing using the RAM 33 or the fixed disk 34 according to the program 811 . The CPU 31 and the GPU 39 function as an arithmetic unit in the first computer 3 . In addition to the CPU 31 and the GPU 39 , other configurations that function as the arithmetic unit may be employed.

FIG. 3 is a diagram showing a functional configuration realized by the first computer 3 executing arithmetic processing and the like according to the program 811 . These functional structures include a storage unit 301 , an inspection unit 302 , and a control unit 303 . All or part of these functions can also be dedicated circuit to achieve. In addition, these functions can also be implemented by a plurality of computers. In the functional configuration shown in FIG. 3 , the inspection unit 302 and the control unit 303 are realized by the CPU 31 , the GPU 39 , the ROM 32 , the RAM 33 , the fixed disk 34 , and their peripheral structures. In addition, the storage unit 301 is mainly realized by the RAM 33 and the fixed disk 34 .

The storage part 301 stores the image of the board|substrate 9 acquired by the imaging part 21, the learning completion model mentioned later, and the like. The inspection unit 302 is a classifier that inspects the image of the substrate 9 using the learned model stored in the storage unit 301 and classifies the image according to the presence or absence of defects. The control unit 303 controls the operation of the imaging unit 21 , the stage moving mechanism 23 , and the respective functional structures.

FIG. 4 is a diagram showing the configuration of the second computer 4 . The second computer 4 has substantially the same configuration as the first computer 3 and has a general computer system including a CPU 41 , a ROM 42 , a RAM 43 , a fixed disk 44 , a display 45 , an input unit 46 , and a reading device 47 , the communication part 48 , the GPU 49 , and the bus bar 40 . The CPU 41 performs various arithmetic processing. The GPU 49 performs various arithmetic processing related to image processing. The ROM 42 stores basic programs. The RAM 43 stores various information. The fixed disk 44 stores information. The display 45 displays various information such as images. The input unit 46 includes a keyboard 46a and a mouse 46b that receive input from the operator. The reading device 47 reads information from a computer-readable recording medium 82 such as an optical disk, a magnetic disk, a magneto-optical disk, and a memory card. The display 45, the keyboard 46a, the mouse 46b, and the reading device 47 are connected to the bus bar 40 via the interface I/F. The communication unit 48 transmits and receives signals to and from other components of the inspection apparatus 1 and external devices. The bus bar 40 connects the CPU 41, A signal circuit to which the GPU 49 , the ROM 42 , the RAM 43 , the fixed disk 44 , the display 45 , the input unit 46 , the reading device 47 and the communication unit 48 are connected.

In the second computer 4 , the program 821 is read out from the recording medium 82 via the reading device 47 in advance and stored in the fixed disk 44 . The program 821 can also be stored in the fixed disk 44 via a network. The CPU 41 and the GPU 49 execute arithmetic processing using the RAM 43 or the fixed disk 44 according to the program 821 . The CPU 41 and the GPU 49 function as an arithmetic unit in the second computer 4 . In addition to the CPU 41 and the GPU 49 , other configurations that function as the arithmetic unit may be employed.

FIG. 5 is a diagram showing a functional configuration realized by the second computer 4 executing arithmetic processing and the like according to the program 821 . These functional structures include a first input unit 401 , a second input unit 402 , a learning unit 403 , and a storage unit 404 . All or part of these functions can also be implemented by dedicated circuits. In addition, these functions can also be implemented by a plurality of computers. Among the functional structures shown in FIG. 5 , the first input unit 401 , the second input unit 402 and the learning unit 403 are realized by the CPU 41 , the GPU 49 , the ROM 42 , the RAM 43 , the fixed disk 44 and their peripheral structures. In addition, the storage unit 404 is mainly realized by the RAM 43 and the fixed disk 44 .

The first input unit 401 inputs, to the storage unit 404 , the first data, which is a collection of images of the classified substrates 9 (hereinafter, referred to as “substrate images”). The second input unit 402 inputs the second data, which is a set of reclassified substrate images, to the storage unit 404 . The storage unit 404 stores data and the like input from the first input unit 401 and the second input unit 402 . Details of the classification and reclassification of the board images will be described later.

The learning unit 403 generates a learning data set from the second data, and uses the A learning completion model for the inspection by the inspection unit 302 is created using the machine learning of the learning data set. The learning unit 403 performs machine learning on the relationship between the substrate image and the presence or absence of defects with respect to the initial model for inspection, thereby creating a learned model. Details of the learning material set will be described later. The machine learning in the learning unit 403 is performed by, for example, deep learning using a neural network. Learning by the deep learning can be performed using, for example, a residual network (Residual Network, ResNet). In addition, the machine learning can also be performed by methods other than deep learning.

FIG. 6 is a diagram showing the flow of creation of the learning completion model in the inspection apparatus 1 . FIG. 7 is a diagram showing a state in which a learning completed model is created. In the preparation of the learning-completed model, first, many images (ie, substrate images) of various regions selected from the images obtained by photographing the substrate 9 are prepared (step S11 ). In the following description, these many board|substrate images are called "substrate image group". The substrate image group is preferably acquired by the imaging unit 21 of the inspection apparatus 1 . In addition, it is preferable that images obtained by photographing a plurality of substrates 9 are included in the substrate image group.

In the substrate 9 , a protective film called a solder resist is formed on a copper wiring or a base material, and various regions exist on the substrate 9 . For example, on the board|substrate 9, the area|region where the copper plating layer is exposed exists, the area|region where solder resist exists on the copper wiring, the area|region where the solder resist exists on a base material, etc. exist.

As shown in FIG. 8, the area where the copper plating layer is exposed includes a small area plating area 93a with a small area of each exposed plating part, a large area plating area 93c with a large area of each exposed plating part, and an exposed area 93c. The area of each plated portion is a medium-area plated region 93b. In the small area plated area 93a, and the middle area plated area 93b Compared with the large-area plated region 93c, the defect has a greater adverse effect on conduction and the like. In addition, in the medium-area plating region 93b, compared with the large-area plating region 93c, the adverse effects on conduction or the like due to defects are greater. In other words, the importance of the region where the defect is located on the substrate 9 increases in the order of the large-area plating region 93c, the medium-area plating region 93b, and the small-area plating region 93a.

As shown in FIG. 9 , the area covered with the solder resist includes a thin-line solder resist (SR) area 94a in which the wiring under the solder resist is a thin line pattern, and a standard wiring SR area in which the wiring under the solder resist is a standard pattern 94b, and the base material SR region 94c in which only the base material exists without wiring under the solder resist. In the thin line SR region 94a, compared with the standard wiring SR region 94b and the base material SR region 94c, the adverse effects on conduction or the like due to defects are greater. In addition, in the standard wiring SR region 94b, compared with the base material SR region 94c, the adverse effects on conduction or the like due to defects are greater. In other words, the importance of the region where the defect is located on the substrate 9 becomes higher in the order of the base material SR region 94c, the standard wiring SR region 94b, and the thin line SR region 94a.

As shown in FIG. 7 , the prepared group of substrate images (ie, many substrate images) are classified into presumed defective images having defects and presumed good images having no defects. In the second computer 4, the classified board image group is input into the storage unit 404 as the first data through the first input unit 401 (step S12). The classification (ie, inspection) of the substrate image group in step S12 can be performed by various known methods. For example, each substrate image is compared with a reference substrate image that does not have defects by a known visual inspection apparatus, and the substrate images with significant differences are classified as presumed defects. The trapped image is classified, and other substrate images are classified as presumed good images.

Various defect inspections are performed in the classification of the substrate image group in step S12. For example, in the inspection of the plated portion, a plated foreign matter inspection to detect foreign matter or the like on the plated portion, or a plated multi-value inspection to detect the depth of the plating in the plated portion is performed. Defects detected in the plating foreign matter inspection (eg, foreign matter on the plated portion or plating The lack of the cladding portion) has a large adverse effect on conduction, etc. In other words, the importance of the type of inspection becomes higher in the order of the multi-value inspection of plating and inspection of plating foreign objects.

Moreover, for example, in the inspection of a solder resist, the SR peeling test which detects peeling of a solder resist, or the SR spot test which detects the depth of a solder resist is performed. Compared with defects detected in SR spot inspection (eg, bright spots or dark spots of solder resist), defects detected in SR peel inspection (for example, defects of wiring caused by peeling of solder resist) exposure) has a large adverse effect on conduction, etc. In other words, the importance of the type of inspection becomes higher in the order of SR spot inspection and SR peel inspection.

In the first data, the type of the region on the substrate 9 of each hypothetical defect image and the type of inspection in which the defect was detected are combined with each hypothetical defect image. The "inspection area type" and "inspection type" associated with each assumed defect image in the first data are expressed by, for example, text data.

Then, all the hypothetical defect images (hereinafter, also referred to as "hypothetical defect image groups") in the first data stored in the storage unit 404 in step S12 are selected and displayed on the display 45 . Then, as shown in FIG. 7 , an operation of reclassifying the assumed defect image group by the presence or absence of a true defect is performed by visual inspection of the operator. (so-called verification). A true defect refers to a defect that has a substantial adverse effect on the performance of the substrate 9, and therefore it is judged that the substrate 9 cannot be marketed as a product. In the verification process, for example, when the operator judges that there is a real defect by viewing the hypothetical defect image, he right-clicks the hypothetical defect image and selects "True Defect" from the pull-down menu, thereby saving the hypothetical defect image in the True Defects folder. In addition, when the operator judges that there is no real defect by viewing the hypothetical defect image, he right-clicks the hypothetical defect image and selects "false report" from the pull-down menu, thereby saving the hypothetical defect image in the false report folder.

In the second computer 4, the hypothetical defect image group reclassified by the verification is input to the storage unit 404 by the second input unit 402 as the second data (step S13). In the second data, each hypothetical defect image is associated with the inspection area type and inspection type, and the verification result (that is, information indicating the presence or absence of a true defect).

Next, the learning unit 403 generates a learning data set based on the second data (step S14). In step S14, weighting of the assumed defective image is performed in the second data. Specifically, when the assumed defect image is reclassified as having a true defect in step S13, in step S14, one weighting coefficient is selected from a plurality of weighting coefficients prepared in advance, corresponding to the importance of the true defect , and multiplied by the assumed defect image. Then, the assumed defective image multiplied by the weighting coefficient is classified as "defective". On the other hand, when the assumed defective image is reclassified as not having a true defect in step S13, in step S14, the assumed defective image is not multiplied by the weighting coefficient, but is classified as "no defect".

The plurality of weight coefficients correspond to values associated with the assumed defect image. The inspection area type, inspection type, and the like are set in advance and stored in the storage unit 404 . In the present embodiment, the weighting coefficient increases as the importance of the type of inspection area (ie, the area where the true defect is located on the substrate 9 ) increases. In addition, the weight coefficient increases as the importance of the inspection type (that is, the inspection type in which a true defect is detected) becomes higher. The plurality of weight coefficients are set between 0 and 1, for example.

Specifically, for example, when the inspection area type is the small-area plated area 93a, and the inspection type is the plated foreign object inspection, the weighting factor ratio is greater than when the inspection area type is the medium-area plated area 93b, and the inspection type is the plated foreign object inspection The weight coefficient is large. In addition, the weighting coefficient when the inspection area type is the small-area plating area 93a and the inspection type is the plating foreign object inspection is higher than the weighting coefficient when the inspection area type is the small-area plating area 93a and the inspection type is the plating multi-value inspection big. When the inspection area type is the small-area plating area 93a and the inspection type is plating foreign matter inspection, the weighting factor is larger than when the inspection area type is the large-area plating area 93c and the inspection type is plating multi-value inspection.

For example, when the inspection area type is the thin line SR area 94a and the inspection type is SR peeling inspection, the weighting factor is larger than that when the inspection area type is the standard wiring SR area 94b and the inspection type is SR peeling inspection. The weighting factor when the inspection area type is the thin line SR area 94a and the inspection type is SR peeling inspection is larger than that when the inspection area type is the thin line SR area 94a and the inspection type is SR spot inspection. When the inspection area type is the thin line SR area 94a and the inspection type is SR peel inspection, the weighting factor is larger than that when the inspection area type is the substrate SR area 94c and the inspection type is SR spot inspection.

In addition, the weight coefficient may be set in advance corresponding to only one of the inspection area type and the inspection type. In addition, the weight coefficient may be set in advance based on other information associated with the assumed defect image (that is, information other than the inspection area type and inspection type). The weight coefficient does not necessarily need to be set between 0 and 1, and can be appropriately changed.

When the learning data set is generated, the learning unit 403 uses the learning data set to perform machine learning on the relationship between the substrate image and the presence or absence of defects with respect to the initial model for inspection. Thereby, the learning completed model used when the board|substrate 9 is inspected is produced (step S15).

FIG. 10 is a diagram showing a flow of inspection of the board 9 using the learning completion model. In the inspection apparatus 1 shown in FIG. 1, first, as described above, a learning completion model is created by the second computer 4 as a learning apparatus (step S21). The learned model is transmitted from the second computer 4 to the first computer 3 and stored in the storage unit 301 (see FIG. 3 ) of the first computer 3 .

Next, the imaging unit 21 and the stage moving mechanism 23 are controlled by the control unit 303, whereby the substrate 9 is photographed by the imaging unit 21, and an image of the substrate 9 to be inspected (hereinafter, also referred to as "inspected object" is acquired). image") (step S22).

Next, the inspection unit 302 performs inspection of the image to be inspected using the learned model stored in the storage unit 301 (step S23). In step S23, for example, the above-mentioned plating foreign matter inspection, plating multi-value inspection, SR peeling inspection, SR spot inspection, and the like are performed. In addition, among the board images of the regions selected from the inspection images, it is determined that the bases classified as "defective" in the learning completed model are the same. Substrate images that are similar to board images (ie, substrate images with true defects) are classified as defect images. In addition, the substrate images that are not classified as defective images are classified into good product images.

After that, with respect to the board image classified into the defect image by the inspection unit 302, the operator performs verification to determine the presence or absence of a true defect in the board 9 (that is, the board 9) in the same manner as in step S13. whether it can be listed) (step S24). In step S24, the board image classified as a defect image by the inspection unit 302 is reclassified into a board image with a true defect and a board image without a true defect (ie, false report).

In the inspection apparatus 1 , the re-learning of the learned model may be performed using the results of the inspection of the substrate 9 by the inspection apparatus 1 (steps S21 to S24 ). FIG. 11 is a diagram showing a flow of relearning of the learned model in the inspection apparatus 1 . In the re-learning of the learned model, the second input unit 402 (refer to FIG. 5 ) of the second computer 4 firstly uses the second input unit 402 of the second computer 4 (refer to FIG. 5 ) to perform the re-classification of the defective image in the step S24 (that is, in the step S23 ). The defect image obtained in 2 is assumed to be a hypothetical defect image, and the defect image reclassified by the presence or absence of a true defect) is input to the storage unit 404 as new second data (step S31 ).

Next, the learning unit 403 generates a new learning data set based on the new second data (step S32). In step S32, in the same manner as in step S14 described above, weighting of the assumed defect image is performed in the new second data. Specifically, when the assumed defective image is reclassified as having a true defect in step S31, in step S32, from the plurality of weighting coefficients prepared in advance, corresponding to the true defect The importance of the defect is used to select a weighting factor and multiply it with the assumed defect image. Then, the assumed defective image multiplied by the weighting coefficient is classified as "defective". On the other hand, when the assumed defective image is reclassified as not having a true defect in step S31, in step S32, the assumed defective image is not multiplied by the weighting coefficient, but is classified as "no defect".

When the new learning data set is generated, the learning unit 403 relearns the relationship between the substrate image and the presence or absence of defects with respect to the learned model by machine learning using the new learning data set (step S33 ).

As described above, the second computer 4 is a learning device that creates a learned model used when inspecting a board having a pattern on the surface. The learning device includes a first input unit 401 , a second input unit 402 , and a learning unit 403 . The first input unit 401 inputs the first data in which the substrate image is classified into a hypothetical defective image with defects and a hypothetical good image without defects. The second input unit 402 inputs the second data in which the assumed defect images in the first data are reclassified by the presence or absence of true defects. The learning unit 403 performs weighting on the assumed defect image in the second data to generate a data set for learning. The learning unit 403 learns the relationship between the substrate image and the presence or absence of defects by machine learning using the learning data set, and generates a learned model.

In the generation of the learning data set by the learning unit 403, when it is assumed that the defective image is reclassified as having a true defect, among the plurality of weighting coefficients, the weighting coefficient selected according to the importance of the true defect is After multiplying the hypothetical defect images, the hypothetical defect images are classified as "defective". In addition, when it is assumed that the defective image is reproduced When classified as not having a true defect, the assumed defect image is classified as "no defect".

The inspection of the substrate 9 is carried out by using the learning completion model created by using the learning data set, whereby a true defect having a high importance is compared with an image similar to an image of a substrate having a true defect having a low importance. A substrate image similar to an image is easily detected as a defect image. As a result, it is possible to reduce an image that is similar to an image of a substrate having a true defect of low importance, but does not actually have a true defect (ie, an image of the substrate that is judged to be false in the verification), and the inspection of the substrate 9 can be reduced. probability of being detected as a defective image. In other words, the false alarm rate in the defect inspection of the substrate 9 can be reduced, and the inspection with high precision can be realized.

In addition, as described above, in the generation of the learning data set by the learning unit 403, the hypothetical defect image having the true defect is multiplied by the weighting coefficient corresponding to the importance of the true defect, whereby the hypothetical defect can be reduced. The number of images (that is, the capacity of the assumed defect image group) is generated, and a learning data set for making a learning completion model that can realize the above-mentioned high-precision inspection is generated. That is, in the learning apparatus, the capacity of the data set for learning can be reduced, and the inspection of the board 9 with high precision can be realized.

As described above, it is preferable that the plurality of weighting coefficients become larger as the importance of the region where the true defect is located on the substrate 9 becomes higher. Thereby, the defects in the regions with high importance (ie, high-sensitivity regions) on the substrate 9 can be preferentially detected compared to the defects in the regions with low importance. As a result, the false alarm rate in the defect inspection of the substrate 9 can be further reduced, and the inspection with higher precision can be realized.

As described above, it is preferable that the plurality of weighting coefficients increase as the importance of the type of inspection for detecting a true defect among various inspections increases. Thus, with less important Defects in inspections of high importance (that is, inspections with high fatality) can be preferentially detected over defects in inspections of the kind. As a result, the false alarm rate in the defect inspection of the substrate 9 can be further reduced, and the inspection with higher precision can be realized.

As described above, the inspection apparatus 1 is an apparatus for inspecting the substrate 9 having a pattern on the surface. The inspection apparatus 1 includes the above-described learning apparatus and the inspection unit 302 . The inspection unit 302 inspects the image to be inspected obtained by photographing the substrate 9 using the learned completed model created by the learning device. Thereby, as described above, the inspection of the substrate 9 with high accuracy can be realized.

As described above, it is preferable that the second input unit 402 inputs new second data reclassified by the presence or absence of a true defect, using the defect image obtained by the inspection unit 302 as a hypothetical defect image. In addition, the learning unit 403 performs weighting on the assumed defect image in the new second data, and generates a new data set for learning. Then, the learning unit 403 relearns the relationship between the board image and the presence or absence of defects with respect to the learned model by machine learning using the new learning data set. Thereby, the capacity of the learning data set for relearning can be reduced, and the inspection accuracy of the substrate 9 can be improved.

In addition, in the inspection device 1, if the inspection unit 302 inspects the image to be inspected using the learned model, it is not necessarily necessary to install a learning device. In this case, the inspection unit 302 inspects the image to be inspected obtained by photographing the substrate 9 using a learned model created by a learning device provided independently of the inspection device 1 . Also in this case, it is possible to realize the inspection of the substrate 9 with high accuracy in the same manner as described above.

As mentioned above, it is preferable that the inspection device 1 further includes a photographing substrate 9 to acquire The imaging unit 21 of the image to be inspected. Thereby, the acquisition of the image to be inspected to the inspection can be performed continuously by one device. As a result, the inspection efficiency of the substrate 9 can be improved.

As described above, the learning method for making the learning completion model includes: inputting first data (step S12 ) in which the substrate image is classified into a presumed defective image with defects and a presumed good image without defects (step S12 ); The presence or absence of defects is a step of reclassifying the assumed defect images in the first data to the second data (step S13); and performing weighting on the presumed defect images in the second data to generate a learning data set, The steps of creating a learning completion model by learning the relationship between the substrate image and the presence or absence of defects by machine learning using the learning data set (step S14 to step S15 ).

In the generation of the learning data set in steps S14 to S15, when it is assumed that the defective image is reclassified as having a true defect, among the plurality of weighting coefficients, the weight selected according to the importance of the true defect is set. After the coefficients are multiplied by the hypothetical defect image, the hypothetical defect image is classified as defective, and when the hypothetical defect image is reclassified as not having a true defect, the hypothetical defect image is classified as defect-free. As a result, the capacity of the learning data set can be reduced in the same manner as described above, and the inspection of the board 9 with high precision can be realized.

Various modifications can be made to the above-described learning device, inspection device 1 , learning method, and inspection method.

For example, the substrate image group prepared in step S11 may be acquired outside the inspection apparatus 1 and input to the inspection apparatus 1 via a recording disk or a network. Other In addition, in step S22 , the image to be inspected may be acquired from the outside of the inspection apparatus 1 in the same manner, and input to the inspection apparatus 1 . In this case, the imaging unit 21 may be omitted from the inspection apparatus 1 .

The inspection performed on the substrate 9 in the steps S12 and S23 is not limited to the above-mentioned plating foreign matter inspection, SR peeling inspection, and the like, and various inspections can be performed on the substrate 9 . In addition, the inspection area on the substrate 9 is not limited to the above-mentioned small-area plating area, the thin line SR area, and the like, and various areas can be set on the substrate 9 .

In the generation of the learning data set in the step S14, the weight coefficient by which the assumed defect image is multiplied does not necessarily need to be set in advance, and the operator may visually inspect the assumed defect image and determine it based on experience during the verification of the step S13, etc. set up. The same is true for the relearning of the learning completion model (step S31 to step S33 ) using the inspection result obtained by the inspection unit 302 .

In the re-learning of the learning completion model (steps S31 to S33), in the generation of the new learning data set in step S32, the weighting of the assumed defect image (that is, the importance corresponding to the defect may not be performed. multiplication of the weight coefficients of the degree). Even in this case, by completing the relearning of the model by learning, the inspection accuracy of the substrate 9 can be improved. In addition, it is not always necessary to perform the re-learning of the learning-completed model (step S31 to step S33).

The first computer 3 and the second computer 4 can also be accommodated in the frame of the device body 2 . Conversely, the second computer 4 can also be used as a single learning device independent of other structures of the inspection device 1 .

The configurations in the above-described embodiment and each modification are not contradictory to each other. Can be combined appropriately.

Although the present invention has been described and described in detail, the described description is merely illustrative and not intended to limit the present invention, and many modifications and aspects can of course be incorporated without departing from the scope of the present invention.

4: Second computer

45: Display

401: First input part

402: Second input part

403: Learning Department

404: Storage Department

Claims (16)

A learning device is a learning device for producing a learning completion model used when inspecting a substrate having a pattern on the surface, comprising: a first input part, wherein the input substrate image is classified into a hypothetical defect image with defects and an image without a defect. a first data of assumed good images of defects; a second input unit for inputting second data for reclassifying the presumed defective images in the first data by the presence or absence of true defects; and a learning unit for inputting In the second data, weighting the assumed defect image is performed to generate a learning data set, and by machine learning using the learning data set, the relationship between the substrate image and the presence or absence of defects is learned, and the learning is completed. A model wherein, in the generation of the learning data set by the learning unit, when the assumed defect image is reclassified as having a true defect, a plurality of weighting coefficients are set corresponding to the true defect. After the weight coefficient selected by the importance of the defect is multiplied by the assumed defect image, the assumed defect image is classified as defective, and when the assumed defect image is reclassified as not having a true defect, the assumed defect image is classified as having no real defect. The hypothetical defect image is classified as non-defective. The learning apparatus according to claim 1, wherein the plurality of weighting coefficients become larger as the importance of the region where the true defect is located on the substrate becomes higher. The learning apparatus according to claim 1 or claim 2, wherein the plurality of weighting coefficients increase as the importance of the type of inspection in which the true defect is detected among the plurality of inspections becomes higher. An inspection device is an inspection device for inspecting a substrate having a pattern on the surface, comprising: an inspection part using the inspection part produced by the learning device according to any one of claim 1 to claim 3 After learning the completed model, the inspection image obtained by photographing the substrate is inspected. The inspection apparatus according to claim 4, further comprising: an imaging unit that captures the image to be inspected by imaging the substrate. An inspection device is an inspection device for inspecting a substrate having a pattern on the surface, comprising: the learning device according to any one of claim 1 to claim 3; The produced learning completed model is inspected by the inspection image obtained by photographing the substrate. The inspection device according to claim 6, wherein the second input unit of the learning device inputs a defect image obtained by the inspection unit as the hypothetical defect image; Whether there is new second data that has been reclassified, the learning unit performs weighting on the assumed defect image in the new second data to generate a new learning data set, and uses the new data set. For the machine learning of the learning data set, the relationship between the substrate image and the presence or absence of defects is re-learned for the learning completion model. The inspection apparatus according to any one of claim 6, further comprising: an imaging unit that captures the image to be inspected by imaging the substrate. A learning method is a learning method for making a learning completion model used when inspecting a substrate having a pattern on the surface, comprising: a) an input substrate image is classified into a hypothetical defect image with defects and a hypothetical defect image without defects the step of inputting the first data of the good image; b) the step of inputting the second data which reclassifies the assumed defective image in the first data by the presence or absence of true defects; and c) in the first data Steps of generating a learning data set by weighting the assumed defect image in the second data, and creating a learning completion model by learning the relationship between the substrate image and the presence or absence of defects by machine learning using the learning data set , wherein in the generation of the learning data set in the step c), when the hypothetical defect image is reclassified as having a true defect, a plurality of weighting coefficients are set corresponding to the true defect. After multiplying the presumed defective image by the weight coefficient selected by the importance of the presumed defective image, the presumed defective image is classified as defective. The description assumes that defective images are classified as defect-free. The learning method according to claim 9, wherein the plurality of weighting coefficients become larger as the importance of the region where the true defect is located on the substrate becomes higher. The learning method according to claim 9, wherein the plurality of weighting coefficients become larger as the importance of the kind of inspection in which the true defect is detected among the plurality of inspections becomes higher. An inspection method, which is to inspect a substrate with a pattern on the surface An inspection method for inspection includes the step of inspecting an image to be inspected obtained by photographing a substrate using a learning completion model created by the learning method described in any one of Claims 9 to 11. The inspection method according to claim 12, further comprising: photographing the substrate to obtain the inspected image. An inspection method, which is an inspection method for inspecting a substrate having a pattern on the surface, comprising: d) the step of making a learning completion model by the learning method as described in any one of claim 9 to claim 11; and e) A step of inspecting the image to be inspected obtained by photographing the substrate by using the learning completion model produced in the step d). The inspection method according to claim 14, further comprising: f) inputting the defect image obtained in the step e) as the hypothetical defect image, and reclassifying the new image based on the presence or absence of true defects and g) performing weighting on the assumed defect image in the new second data to generate a new learning data set, by using the new learning data set Machine learning, in which the model re-learns the relationship between the substrate image and the presence or absence of defects for the learning. The inspection method according to claim 14, further comprising: photographing the substrate to obtain the inspected image.

Images