JP7054450B2 - Work inspection method - Google Patents

Description

The present invention relates to a work inspection method and a work inspection system, and more particularly to a method and a system for inspecting the presence or absence of defects in a brittle work and determining the quality of the work.

As is well known, in a work manufacturing factory, etc., before the work is manufactured and shipped, the presence or absence of defects in the work is inspected and the quality of the work is judged, and only the work judged to be good is loaded. Is done. Various image processing methods have been conventionally used as a method for determining the quality of this work.

Specifically, the image of the work captured by the imaging means is processed into a necessary image by performing various processes using a computer program, and the quality of the work is judged based on the image. Is being done (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2017-167047

By the way, when the work has brittleness, not only a wide variety of defects but also unpredictable defects may exist as compared with the metal work or the like due to the peculiarity of the manufacturing method or the like. ..

Therefore, defects that may exist in the work having brittleness cannot be properly inspected by the above-exemplified method, which may lead to a situation in which the reliability of the quality determination of the work is impaired.

From the above viewpoint, it is an object of the present invention to appropriately inspect the presence or absence of defects in a work having brittleness and to improve the reliability of the quality determination of the work.

The method according to the present invention, which was devised to solve the above problems, is a work inspection method in which the presence or absence of defects in a brittle work is inspected to determine the quality of the work, and the work is imaged and the work is imaged. Using the pass / fail judgment learning model constructed by extracting the work image and machine learning, the first judgment step of inspecting the extracted work image for defects and making a pass / fail judgment of the work, and machine learning are performed. It is characterized by having a second determination step of inspecting the extracted work image for the presence or absence of defects and determining the quality of the work by using an algorithm program that does not include the work. Here, the "good / bad judgment learning model constructed by machine learning" means a learning model for good / bad judgment of a work constructed by processing using artificial intelligence (AI) or processing using a neural network (). The same applies below). Further, the “algorithm program that does not include machine learning” means an algorithm program that does not include processing using artificial intelligence (AI) or processing using a neural network (hereinafter, the same applies).

According to such a method, the advantages of the first judgment process using machine learning and the second judgment process not using machine learning are effectively utilized, and the presence or absence of defects in the brittle workpiece is appropriately inspected. Therefore, it is possible to improve the reliability of the quality judgment of the work. In detail, as described above, since a work having brittleness may have a wide variety of defects and unpredictable defects, it is difficult to obtain a highly reliable determination result only by a single determination step. be. However, if the two determination steps having different advantages as in the present invention are provided, the reliability of the quality determination of the brittle work can be sufficiently enhanced.

In this case, the first determination step may be performed before the second determination step, and when the work is determined to be defective in the first determination step, the second determination step may not be performed. .. Therefore, when the work is determined to be good in the first determination step, the second determination step is performed, and the result of the pass / fail determination in the second determination step is prioritized.

By doing so, the time required for the quality determination of the work is effectively shortened and the productivity is enhanced despite having two determination steps. In addition, even if a judgment error occurs in the first judgment process, which is good or bad, from the standpoint of giving top priority to collecting and shipping only good products, reliability and quality assurance are achieved. Will be excellent.

The second determination step may be performed before the first determination step, and when the work is determined to be good in the second determination step, the first determination step may be further performed. In this case, if the work is determined to be defective in the second determination step, the first determination step may or may not be further performed. Further, the first determination step and the second determination step may be performed at the same time. In this case, if the result of the pass / fail judgment in the first judgment step is bad, the result is given priority, and if the result of the pass / fail judgment in the second judgment step is good, the result is given priority. You may do it.

In the above method, in the first determination step, the probability that the work is good is calculated, and the result of the quality determination of the work is determined based on the probability and a predetermined threshold value for the probability. May be obtained.

By doing so, not only "good" or "bad" can be obtained as a judgment result, but also a carefully calculated judgment result can be obtained, so that the reliability of the quality judgment in the first judgment step is further improved. It will be further enhanced.

In the above method, the front surface and the back surface of the work are the target surfaces for defect inspection, the work is placed on a transparent substrate, and the work is imaged from the front side to obtain one work image and at the same time. Another work image may be obtained by imaging the work from the back side via the transparent substrate.

By doing so, the work image can be extracted from the front surface and the back surface of the work without turning over the work, so that the first determination step and the second determination step can be performed quickly and efficiently.

In the above method, the front surface and the back surface of the work are the target surfaces for defect inspection, the work is placed on a transparent substrate, and the work is imaged from the front side to construct the pass / fail judgment learning model. By obtaining one learning work image used for the purpose and imaging the work from the back side via the transparent substrate, another learning work image used for constructing the pass / fail judgment learning model can be obtained. You may.

By doing so, the number of learning work images used for constructing the pass / fail judgment learning model can be doubled. Moreover, it is possible to obtain two different types of learning work images, one is a learning work image captured and extracted through a transparent substrate, and the other is a learning work image captured and extracted without a transparent substrate. .. Therefore, it is possible to construct a good / bad judgment learning model capable of performing good / bad judgment of the work with high accuracy by using a large number of various learning work images.

In this case, one modification in which at least one of the learning work image of the one learning work image and the other learning work image obtained for use in constructing the pass / fail judgment learning model is rotated by a predetermined angle. At least one of the processing and the other modification processing in which at least one of the learning work image of the one learning work image and the other learning work image is inverted on both sides of the straight line passing through the central portion thereof. By performing the modification process, a new learning work image may be created, and the pass / fail judgment learning model may be constructed using the new learning work image and the learning work image before the modification process. ..

By doing so, the number of learning work images used for constructing the pass / fail judgment learning model can be further increased by a simple process. Therefore, it is possible to construct a pass / fail judgment learning model that can perform the pass / fail judgment process of the work with higher accuracy.

In the above method, in the first determination step, at least one of the chipped edge of the work, the burr protruding in the thickness direction of the edge of the work, and the black spot foreign matter scattered and adhered to the work is recognized as the defect of the work. It is preferable to do so.

By doing so, a special abnormal portion that may occur due to the brittleness of the work is recognized as a defect, which leads to improvement in the reliability of the pass / fail judgment in the first judgment step.

In this case, it is preferable to recognize whether or not the black spot foreign matter scattered and adhered to the work is a defect depending on the size, number, and dense state of the black spot foreign matter.

In this way, not all of the black spot foreign substances scattered and adhered to the above-mentioned work are recognized as defects, so that the defects are recognized precisely and the quality is judged in the first judgment step. It leads to the improvement of reliability.

In the first determination step, dust and dirt adhering to a part of the work can be prevented from being recognized as defects in the work. Therefore, it is less likely that a determination error of recognizing dust or dust that is not a defect as a defect is likely to occur, which leads to improvement in the reliability of the quality determination in the first determination step.

In the above method, when the result of the quality determination of the work obtained from the first determination step and the second determination step is defective, the defect classification learning model constructed by machine learning is used to determine the defect. A defect classification step may be performed to classify which of the types of defects classified into a plurality of types in advance the defects in the work image belong to. Here, the "defect classification learning model constructed by machine learning" means a learning model for defect classification constructed by processing using artificial intelligence (AI) or processing using a neural network (hereinafter,). Similarly).

By doing so, each time the defect classification process is performed, the cumulative number of defects for each of the types classified into a plurality of types is known, so that the reason for the occurrence of defects at the time of manufacturing the work can be known. Therefore, the occurrence of defects can be reduced by changing the manufacturing conditions and the like according to the reason for the occurrence of the defects. Moreover, since the defect classification process is performed using the defect classification learning model constructed by machine learning, the reliability of the process of classifying defects into a plurality of types is improved.

In this method, a plurality of defective learning work images are selected from a plurality of learning work images used when constructing the pass / fail judgment learning model, and defects are selected from the defective learning work images. A plurality of learning defect images as partial images including only the defect image and its peripheral region may be extracted, and the defect classification learning model may be constructed by using the learning defect images.

By doing so, the learning work image used when constructing the pass / fail judgment learning model is effectively used for constructing the defect classification learning model, and the image processing can be simplified.

In this case, one modification process of rotating the learning defect image obtained for use in constructing the defect classification learning model by a predetermined angle, and inverting the learning defect image on both sides of a straight line passing through the central portion thereof. A new learning defect image is created by performing at least one of the other modification processes to be performed, and the new learning defect image and the learning defect image before the modification process are used as described above. A defect classification learning model may be constructed.

By doing so, the number of learning work images used for constructing the defect classification learning model can be increased by a simple process. Therefore, it is possible to construct a defect classification learning model that can perform defect classification processing with even higher accuracy.

In the above method, in the defect classification step, it is preferable to classify the crack generated in a part of the work and the black foreign matter adhering to the part of the work as different types of defects.

In this way, when classifying defects into a plurality of types, it is less likely that a classification error will occur in which the above two types of defects are classified into the same type of defects, and the reliability of processing in the defect classification process is improved. Be enhanced.

Further, in the defect classification step, it is preferable to classify the white or black three-dimensional foreign matter adhering to a part of the work and the planar foreign matter having the same color as the three-dimensional foreign matter as different types of defects. ..

In this way, when classifying defects into a plurality of types, it is less likely that a classification error will occur in which the above two types of defects are classified into the same type of defects, and the reliability of processing in the defect classification process is improved. Be enhanced.

In the above method, when the correct answer rate as a result of the pass / fail judgment in the first determination step decreases, all the defect classification steps performed before the correct answer rate starts to decrease are classified into the plurality. The pass / fail determination learning model may be updated by machine learning using a defective work image having a defect that does not belong to any of the types of defects as a learning work image.

Here, the present inventors classify the reason why the correct answer rate of the result of the pass / fail judgment decreases in the first determination step into a plurality of reasons in all the defect classification steps performed before the correct answer rate starts to decrease. It was found that this was caused by the appearance of defective work images with defects that did not belong to any of the types of defects that were made. Therefore, it was devised to update the pass / fail judgment learning model by effectively utilizing such a defective work image. Then, by updating this pass / fail judgment learning model, judgment errors are less likely to occur, and the above-mentioned correct answer rate is restored, so that it is possible to prevent a decrease in the reliability of the pass / fail judgment.

In the above method, the work may be a tablet-shaped ceramic sintered product, and the work image may be an image of the work viewed in the thickness direction.

In this way, it is less likely that a pass / fail judgment error or a defect classification error will occur for special defects or unpredictable defects that may exist on the front surface or the back surface of the tablet-shaped ceramic sintered product.

In the above method, the machine learning algorithm is preferably deep learning.

By doing so, the above-mentioned pass / fail judgment learning model and defect classification learning model can be constructed with high accuracy, and the reliability of the first judgment step and the defect classification process can be ensured.

In the above method, a work image can be obtained by taking an image of the work in a state where the work is illuminated with ring illumination. Here, the ring illumination has a characteristic that it is more responsive to unevenness as compared with other illuminations such as dome illumination. Therefore, if the work image is extracted using the ring illumination, the convex defects and the concave defects existing in the work can be clearly recognized. Therefore, convex defects and concave defects that were difficult or impossible to recognize when other lighting was used can be recognized, and the types of recognizable defects increase, and the quality of the work is improved. It is less likely that mistakes will occur in the judgment. Moreover, the types of convex defects and concave defects can be discriminated without mistakes.

Further, in the above method, a learning work used for constructing the learning model (pass / fail judgment learning model and defect classification learning model) by imaging the work in a state where the work is illuminated with ring illumination. You can also get an image. In this case, a work image having more types of defects can be used as a learning work image as compared with the case where other lighting is used. This makes it possible to construct a learning model capable of determining the quality of the work and classifying the defects with high accuracy.

The system according to the present invention, which was devised to solve the above problems, is a work inspection system that inspects the presence or absence of defects in a brittle work and determines the quality of the work, and is an imaging means for imaging the work. Then, using the work image extraction processing unit that extracts the work image from the image captured by the imaging means and the pass / fail judgment learning model constructed by machine learning, the extracted work image is inspected for defects. The first judgment processing unit that judges the quality of the work and the algorithm program that does not include machine learning are used to inspect the extracted work image for defects and make a second judgment to judge the quality of the work. It is characterized by having a processing unit.

Since this system has substantially the same configuration as the corresponding method described above although the category is different, substantially the same operation and effect as the method can be obtained.

In the above method and system, in the pass / fail judgment learning model, the first convolution layer, the pooling layer, the second convolution layer, and the connection layer are arranged between the input layer and the output layer in order from the input layer side. It is preferable that the learning model is based on a four-layer neural network including a dropout layer.

In this case, it is preferable that the above-mentioned neural network has a processing layer for randomly processing a learning work image on the input layer side of the above-mentioned first convolution layer.

Further, in the above-mentioned neural network, it is preferable that the number of neurons in the connecting layer existing on the input layer side is 512 or less.

Further, the above-mentioned neural network preferably has a filter size of 5 or more.

With the configuration of the neural network as described above, it is possible to obtain a pass / fail judgment learning model capable of performing the pass / fail judgment process of the work with high accuracy.

In the above method and system, the defect classification learning model has three layers, including eight convolutional layers, a coupling layer, and a softmax layer, in order from the input layer side, between the input layer and the output layer. It is preferable that the learning model is based on a neural network. Further, this neural network preferably has a filter size of 3 or more.

With such a neural network configuration, it is possible to obtain a defect classification learning model capable of performing defect classification processing with high accuracy.

The method and system according to the present invention may abolish the second determination step (second determination processing unit), and even in such a case, the work having brittleness can be solved by solving the above problems. It is possible to increase the reliability of the quality judgment.

According to the present invention, it is possible to appropriately inspect the presence or absence of defects in a work having brittleness and improve the reliability of the quality determination of the work.

It is a perspective view which shows the work which has a brittleness used for carrying out the work inspection method which concerns on embodiment of this invention. It is a schematic front view which shows the work inspection system which concerns on embodiment of this invention. 3A, 3B, and 3C are front views showing images in the process of extracting work images in the work inspection method according to the embodiment of the present invention, respectively. It is a schematic diagram which shows each process about the quality determination in the work inspection method which concerns on embodiment of this invention. It is a flowchart which shows the procedure which performs the pass / fail judgment in the 1st judgment step in the work inspection method which concerns on embodiment of this invention. It is a flowchart which shows the procedure in the case of constructing the pass / fail judgment learning model used in the first judgment step in the work inspection method which concerns on embodiment of this invention. 7 (a), (b), (c), and (d) are front views showing work images to be subjected to quality determination in the work inspection method according to the embodiment of the present invention, respectively. It is a flowchart which shows the procedure in the case of updating the pass / fail judgment learning model used in the first judgment step in the work inspection method which concerns on embodiment of this invention. It is a flowchart which shows the procedure which performs the pass / fail judgment in the 2nd judgment step in the work inspection method which concerns on embodiment of this invention. It is a flowchart which shows the procedure which performs the defect classification process in the work inspection method which concerns on embodiment of this invention. It is a flowchart which shows the procedure in the case of constructing the defect classification learning model used in the defect classification process in the work inspection method which concerns on embodiment of this invention. 12 (a) and 12 (b) are front views showing defect images to be subjected to the defect classification step in the work inspection method according to the embodiment of the present invention, respectively. 13 (a) and 13 (b) are front views showing defect images to be subjected to the defect classification step in the work inspection method according to the embodiment of the present invention, respectively. 14 (a) and 14 (b) are front views showing defect images to be subjected to the defect classification step in the work inspection method according to the embodiment of the present invention, respectively. It is a schematic block diagram which shows the work inspection system which concerns on embodiment of this invention in more detail.

Hereinafter, the work inspection method and the work inspection system according to the embodiment of the present invention will be described with reference to the attached drawings.

FIG. 1 illustrates a work W having brittleness, which is an object to be inspected in this embodiment. As shown in the figure, the work W has a tablet shape. Specifically, this work W is a ceramic sintered product in which a circular through hole Wh is formed in the central portion of a disk-shaped body or a columnar body. More specifically, this work W is, for example, PbO-SiO ₂ -B ₂ О ₃ system glass or SiO ₂ -Al ₂ О ₃ -B ₂ О ₃ -R ₂ О system (R ₂ О is an alkali metal oxidation. (Product) Glass powder made of glass, crystalline bismuth-based glass, etc. and filler powder are mixed and fired. The entire surface of the work W including the front surface W1 and the back surface W2 is a rough surface having minute irregularities, and the front surface W1 and the back surface W2 are the target surfaces for defect inspection.

FIG. 2 is a schematic configuration diagram showing a work inspection system according to this embodiment. As shown in the figure, the work inspection system 1 includes an image pickup means 2. The image pickup means 2 captures the front surface W1 and the back surface W2 of the work W placed on the transparent substrate (for example, a glass substrate) 3 from the front side and the back side of the work W, respectively. Have. The ring illumination 6 is directly applied to the front surface W1 of the work W, and the ring illumination 7 is applied to the back surface W2 of the work W via the transparent substrate 3. The front surface W1 of the work W is imaged by the upper camera 4, and the back surface W2 of the work W is imaged by the lower camera 5 via the transparent substrate 3.

Further, the work inspection system 1 inputs information and data required by the operator to the processing unit 8 that performs various processing based on the images captured by the two cameras 4 and 5 and the processing unit 8. An input unit 9 for processing and a display unit 10 for displaying the results of various processes by the processing unit 8 are provided. The processing unit 8 has an electronic information processing device such as a computer device having a built-in storage means for storing various information and data, as well as a computer program for performing various processes and procedures. For example, the electronic information processing apparatus includes a control means (including an image processing means) such as a CPU for executing a computer program, a storage means such as a memory and a hard disk, and the like. The input unit 9 can be configured by a touch panel, a keyboard, a mouse, or the like operated by an operator. The input unit 9 may be a component of the above-mentioned electronic information processing device, or may be an independent dedicated input device or the like. The display unit 10 can be composed of a display device, a monitor screen, or the like. The display unit 10 may also be a component of the above-mentioned electronic information processing device, or may be an independent dedicated display device or the like. Then, the processes and procedures in each step in the flowcharts shown in FIGS. 5, 6 and 8 to 11 described later are carried out by the above-mentioned processing unit 8, input unit 9 and display unit 10.

Next, the work inspection method according to this embodiment will be described in detail. FIG. 3 is a front view showing a process of extracting a work image from the images captured by the two cameras 4 and 5, respectively. Specifically, the image G1 shown in FIG. 3A is an image captured by the camera 4 (5), and the peripheral region X of the work W is wide. As shown in FIG. 3B, the image G2 in which the peripheral region X of the work W is narrowed is cut out from the image G1. Then, by reducing the image G2 as a whole, the image G3 shown in FIG. 3C can be obtained. The above processing is performed by the image processing means provided in the processing unit 8. The image G3 shown in FIG. 3C is a work image (whole image of the work). Since the work image G3 is a digital image and has a small number of pixels, for example, 200 to 300, image processing described later can be easily performed.

FIG. 4 is a schematic view showing a main process in the case of performing quality determination in the work inspection method. As shown in the figure, the work inspection method includes an imaging step 11 for imaging the front surface W1 and the back surface W2 of the work W by the imaging means 2, a work image extraction step 12 for extracting the work image G3, and a work image G3. It has a first determination step 13 and a second determination step 14 for performing one determination and a second determination, respectively. The work image extraction step 12 is performed by the procedure shown in FIG. 3 described above.

In the first determination step 13, the quality determination learning model constructed by machine learning is used to inspect the work image G3 for defects and determine the quality of the work W. In the second determination step 14, the work image G3 is inspected for defects using an algorithm program that does not include machine learning, and the quality of the work W is determined. Then, for one work W, basically, the first determination step 13 and the second determination step 14 are performed. For the subsequent work W, each of the above steps 11 to 14 is repeated. The work image G3 extracted a plurality of times by repeating the work image extraction step 12 is stored in the storage means provided in the processing unit 8 each time. In this case, the machine learning algorithm in the first determination step 13 is deep learning.

In the first determination step 13 and the second determination step 14, when it is determined that the work W has at least one defect, the work W is determined to be defective, and when it is determined that the work W has no defect. , The work W is judged to be good. In this embodiment, two work images G3 are extracted for one work W due to the configuration of the image pickup means 2. Therefore, if there is even one defect in one work image G3, the work W is It is judged to be defective. Therefore, when there is no defect in any of the two work images G3, the work W is determined to be good.

FIG. 5 is a flowchart showing each of the above steps in chronological order. As shown in the figure, the imaging step 11 is performed in step S1 of the flowchart, the work image extraction step 12 is performed in step S2, and then the first determination step 13 is performed first in step S3. If the result of the pass / fail determination by the first determination step 13 is defective in step S4, the work W is treated as defective and the second determination step 14 is not performed. On the other hand, if the result of the quality determination by the first determination step 13 is good in step S4, the second determination step 14 is further performed in step S5. Then, if the result of the pass / fail judgment by the second determination step 14 is defective in step S6, the work W is treated as a defect, and if the determination result by the second determination step 14 is good, the work W is processed. Is treated as good.

The operation shown in the above flowchart is an example, and when the second determination step 14 is performed before the first determination step 13 and the result of the pass / fail determination by the second determination step 14 is good, the second determination step 14 is performed. One determination step 13 may be further performed. If the result of the pass / fail judgment by the first judgment step 13 at this time is good, the work W is treated as good, and if the result of the pass / fail judgment by the first judgment step 13 is bad, the work W is treated as good. W is treated as defective. If the result of the quality determination by the second determination step 14 performed earlier is poor, the first determination step 13 may or may not be further performed. Further, the first determination step 13 and the second determination step 14 are performed at the same time, and if the result of the pass / fail determination of the first determination step 13 is defective, the result is prioritized and the second determination step is performed. If the result of the pass / fail determination of 14 is good, the result may be prioritized.

The first determination step 13 further includes a step of calculating the probability that the work W is good. Then, in the first determination step 13, the result of whether the work W is good or bad is obtained based on the probability obtained by the above calculation and the threshold value predetermined for the probability. Specifically, the probability that the work W is good is expressed as a percentage, and the threshold value is set to, for example, 90%. Therefore, if the probability that the work W is good is 90% or more, the work W is considered good, and if the probability is less than 90%, the work W is considered bad.

FIG. 6 is a schematic flowchart showing a procedure for constructing a pass / fail judgment learning model used in the first judgment step 13. As shown in the figure, first, in step S11, a plurality of (for example, 1000 or more) learning work images are collected. These learning work images are obtained by capturing the front surface W1 and the back surface W2 (see FIG. 1) of the plurality of work Ws by the imaging means 2, and extracting them from the captured images. The procedure for extracting the learning work image is the same as the procedure shown in FIG. 3, and the processing is performed by the image processing means of the processing unit 8. Further, the aspect of the extracted learning work image is the same as that of the work image G3 shown in FIG. 3 (c). Then, the plurality of learning work images extracted over a plurality of times are stored in the storage means of the processing unit 8 each time.

In this case, in step S12, a new learning work image is created based on the extracted learning work image. Specifically, a modification process is performed in which all or a part of the plurality of extracted learning work images is rotated in the image plane at angles of 45 °, 135 °, 225 °, and 315 °, respectively. Then, create multiple new learning work images. This modification process is performed on the learning work image extracted from at least one of the front surface W1 and the back surface W2 of the work W. In addition, instead of or in combination with this, a modification process of inverting all or a part of the above-mentioned extracted plurality of learning work images on both sides of a straight line passing through the center thereof is performed to change the inclination angle of the above-mentioned straight line. By changing to multiple, multiple new learning work images are created. This modification process is also performed on the learning work image extracted from at least one of the front surface W1 and the back surface W2 of the work W.

Next, in step S13, the collected plurality of learning work images and the newly created learning work image are classified into a good learning work image and a bad learning work image. This classification is performed by the operator operating the input unit 9 while looking at the learning work image displayed on the display unit 10. Here, a good learning work image corresponds to a defect because there is no defect candidate in the work image, or even if there is a defect candidate in the work image, it is small. It is a work image that does not cause the work W to become defective. Further, the defective learning work image is a work image in which at least one defect candidate is present in the work image, at least one of them corresponds to a defect, and the work W becomes defective.

Then, in step S14, a good / bad learning work image is used and a good / bad learning work image is used to construct a good / bad judgment learning model by learning by deep learning. This pass / fail judgment learning model is a learning model using a neural network. A calculation formula is created by constructing this pass / fail judgment learning model. Therefore, strictly speaking, in the first determination step 13, the quality of the work W is determined using this calculation formula. Here, the neural network has a first convolution layer, a pooling layer, a second convolution layer, a connection layer, and a dropout layer between the input layer and the output layer in order from the input layer side. It has 4 layers including. Further, this neural network has a processing layer for randomly processing a learning work image on the input layer side of the above-mentioned first convolution layer. Further, in this neural network, the number of neurons in the connecting layer existing on the input layer side is 512 or less, and the filter size is 5 or more.

Here, with 6067 good learning work images and 4500 bad learning work images, a good / bad judgment learning model is constructed by learning by deep learning, and then the first judgment step 13 is performed for 2000 work Ws. Was done. As a result, there were 17 works W that were determined to be good because they could not be recognized as defects in the past, but were accurately determined to be defective. Representative examples of the work W that has made this accurate determination are shown in FIGS. 7 (a) to 7 (d).

In the work image G3 (1) shown in FIG. 7A, a chip Z1 is generated in the inner peripheral edge portion of the through hole Wh of the work W. In the work image G3 (2) shown in FIG. 7B, burrs Z2 protruding in the thickness direction are generated on the outer peripheral edge of the work W. In the work image G3 (3) shown in FIG. 7 (c), a burr Z3 protruding in the thickness direction is generated in the inner peripheral edge portion of the through hole Wh of the work W. The above-mentioned chipped Z1 in the inner peripheral edge portion, the burr Z2 protruding in the thickness direction of the outer peripheral edge portion, and the burr Z3 protruding in the thickness direction of the inner peripheral edge portion are all accurately recognized as defects. Further, in the work image G3 (4) shown in FIG. 7 (d), black spot foreign matter Z4 is scattered and adhered to the work W. In the embodiment shown in FIG. 7D, whether or not the black spot foreign matter Z4 is defective is recognized according to the size, number, and dense state of the black spot foreign matter Z4, but in the illustrated embodiment, it is accurately recognized as a defect. ing. These defects are unpredictable defects because the work W is brittle and is particularly a ceramic sintered product. Therefore, it has not been possible in the past to recognize these as defects, but the results of the present inventors focusing on these defects and using the above-mentioned pass / fail judgment learning model while conducting intensive research. , These can now be accurately recognized as defects.

Further, in the past, there has been a determination error in which dust or dust adhering to a part of the work W is recognized as a defect. However, in the present invention, even if dust or dirt adheres to the work W as a result of performing the first determination step 13 for 2000 works as described above, the work image G3 of the work W shows dust. And dust are no longer recognized as defects. Again, as a result of the present inventors focusing on dust and dust and using the above-mentioned pass / fail judgment learning model while conducting intensive research, it is possible to eliminate the judgment error of recognizing dust and dust as defects. I can now do it.

While the first determination step 13 is repeated, the accuracy rate of the pass / fail determination in the first determination step 13 may decrease. Here, in this embodiment, whether or not the quality determination is correct is determined by the operator, and the operator inputs the determination result to the processing unit 8 through the input unit 9. The processing unit 8 can aggregate the determination results to calculate the correct answer rate, and display the correct answer rate on the display unit 10. In this case, the processing unit 8 can determine to what extent the correct answer rate has decreased. Therefore, when the processing unit 8 finds that the correct answer rate is lower than the preset predetermined value, it is preferable to interrupt the first determination step 13. The interruption of the first determination step 13 can be performed by a command from the processing unit 8. It should be noted that the determination that the correct answer rate has fallen below the predetermined value and the interruption of the first determination step 13 may be performed by the operator. Here, the present inventors have found that this decrease in the correct answer rate is caused by the suddenly appearing defective work image G3. The defective work image G3 that suddenly appears here refers to any of the types of defects classified into a plurality of defects in all the defect classification steps (described later) performed before the accuracy rate starts to decrease. It also means a defective work image G3 having a defect that did not belong to it. Therefore, based on this finding, the present inventors should update the pass / fail judgment learning model using a defective work image having a type of defect that did not exist before the accuracy rate began to decrease. bottom.

FIG. 8 is a schematic flowchart showing an example of a procedure for updating the pass / fail judgment learning model. As shown in the figure, first, in step S21, a plurality of suddenly appearing defective work images are collected as learning work images. This learning work image is stored in the storage means of the processing unit 8 while the first determination step 13 is being performed, and the operator sees the learning work image displayed on the display unit 10. Therefore, it is collected after confirming whether it is a defect that appears suddenly. In this case, in step S22, a new learning work image is created based on the collected learning work image. Specifically, by performing a modification process in which all or part of the collected work images for learning are rotated in the image plane at intervals of 45 °, 135 °, 225 °, and 315 °, respectively. Create multiple learning work images. Further, instead of or in combination with this, a modification process of inverting the collected learning work image on both sides of a straight line passing through the central portion thereof is performed by changing the inclination angle of the straight line to a plurality. , Create multiple new learning work images. The creation of this new learning work image is performed by the image processing means of the processing unit 8. Next, in step S23, the pass / fail judgment learning model is updated by learning by deep learning using the collected learning work image and the newly created learning work image. Then, this updated pass / fail determination learning model (strictly speaking, a calculation formula) is used when the interrupted first determination step 13 is performed again.

It should be noted that the above-mentioned update of the pass / fail judgment learning model is performed in the same procedure as above, including not only the work image G3 having a defect that suddenly appeared but also the work image G3 having a mistake in the pass / fail judgment other than that. You may do so. Further, such a pass / fail judgment learning model may be updated periodically.

FIG. 9 is a flowchart showing a schematic procedure of pass / fail judgment by an algorithm program that does not include machine learning used in the second judgment step 14. This pass / fail judgment is performed after the number of types of defects existing in the work is known by the defect inspection of the work performed in advance. Then, the modes of the plurality of types of defects are stored in advance in the storage means of the processing unit 8 as binarized images. The figure illustrates the case where there are nine types of defects. The imaging of the work W in step S31 shown in the figure and the extraction of the work image G3 in step S32 have already been performed in the imaging step 11 and the work image extraction step 12 (see FIG. 4). After that, in steps S33 to S41, it is sequentially inspected whether or not the work image G3 has a defect corresponding to the first to ninth types of defects. Here, the types of the first to ninth defects are, for example, chips, cracks, burrs, black foreign substances, white foreign substances, burnt residue, powder, fibers, and dents. Then, when it is determined in any one of steps S33 to S41 that a defect of the corresponding type exists, the work W is treated as a defect. Then, in any step, the work W is treated as good only when it is determined that the defect of the corresponding type does not exist. The defect inspection of the work image G3 here is performed by binarizing the work image G3 and making a determination using a threshold value while comparing it with the binarized image stored in the storage means of the processing unit 8. .. When the second determination step 14 is performed a plurality of times, it is not necessary to inspect all types of defects in most cases, so that the processing can be speeded up and the work efficiency is improved. do.

FIG. 10 is a schematic flowchart showing a post-processing procedure when the result of the quality determination of the work W obtained from the first determination step 13 and the second determination step 14 is defective. As shown in the figure, when the work W is determined to be defective, the work image G3 determined to be defective is taken in from the two work images G3 for one work W in step S41. The defective work image G3 was present in the work image G3 that had already been extracted in the work image extraction step 12 of FIG. 4 and was stored in the storage means of the processing unit 8. Next, in step S42, a defect image is extracted from the captured work image G3. This defect image is a partial image consisting of only the defect and its surrounding area. Specifically, this defect image is an image as shown by the reference numerals G4 (1) to G4 (6) in FIGS. 12 to 14 described later, and the defects Z5 to Z10 are the respective defect images G4 (1). )-G4 (6) is cut so that it exists in the center. Whether or not the work W here is defective is determined by the control means of the processing unit 8, and the defect image is extracted by the image processing means of the processing unit 8.

Then, in step S43 of FIG. 10, using the defect classification learning model constructed by learning by deep learning, the defects existing in the defect image are divided into a plurality (for example, about 10) in advance among the types of defects. Perform a defect classification step to classify which type it belongs to. The types of defects here are chipping, cracking, burrs, black-based three-dimensional foreign matter, black-based planar foreign matter, white-based three-dimensional foreign matter, white-based planar foreign matter, burning residue (black foreign matter), and Examples include powder, fibers, and dents. Then, in step S44, only one defect of the type to which the defect belongs is added. After that, the cumulative number of defects for each of a plurality of types may be displayed on the display unit 10 as a bar graph or the like each time, and after the steps S41 to S44 are repeatedly executed, the display unit 10 may display the cumulative number of defects as a bar graph or the like. You may. Then, by the operator looking at the display unit 10, it is possible to know which type of defect is more and which type of defect is less. By doing so, the worker can recognize which manufacturing condition is bad among various manufacturing conditions for the work W, so that defects occurring in the work W can be reduced.

FIG. 11 is a schematic flowchart showing a procedure for constructing a defect classification learning model used in the defect classification process. As shown in the figure, first, in step S51, a plurality of (for example, 1000 or more) defective work images are collected as learning work images. This defective work image is selected from a plurality of work images G3 stored in the storage means of the processing unit 8 when constructing the above-mentioned pass / fail determination learning model. The worker identifies the defective work image by looking at the work image displayed on the display unit 10. Next, in step S52, learning defect images are extracted and collected from a plurality of learning work images. This learning defect image is a partial image consisting of only the defect and its surrounding area. The aspect of the defect image for learning is the same as the aspect of the image shown by the reference numerals G4 (1) to G4 (6) in FIGS. 12 to 14 described later. The process of extracting the learning defect image is performed by the image processing means of the processing unit 8.

In this case, in step S53, a new learning defect image is created based on the extracted learning defect image. Specifically, a modification process is performed in which all or a part of the plurality of extracted defect images for learning are rotated in the image plane at angles of 45 °, 135 °, 225 °, and 315 °, respectively. Then, create a plurality of new defect images for learning. Further, instead of or in combination with this, a modification process of inverting all or a part of all or a part of the above-mentioned extracted plurality of learning defect images on both sides of a straight line passing through the center thereof is performed by using a plurality of tilt angles of the above-mentioned straight line. By changing to, a plurality of new defect images for learning are created. The creation of this new learning defect image is performed by the image processing means of the processing unit 8.

After that, in step S54, the above-extracted plurality of learning defect images and the newly created plurality of learning defect images are classified into a plurality of types, and a plurality of learning defects are classified for each type. Get an image. The classification here is performed by the operator looking at the learning defect image displayed on the display unit 10. Then, in step S55, a defect classification learning model is constructed by learning by deep learning using a plurality of defect images for learning for each type. This defect classification learning model is a learning model by a neural network. A calculation formula is created by constructing this defect classification learning model. The neural network here includes three layers including an eight convolution layer, a coupling layer, and a softmax layer in order from the input layer side between the input layer and the output layer. Further, in this neural network, the number of neurons in the connecting layer existing on the input layer side is 512 or less, and the filter size is 3 or more.

Here, after constructing a defect classification learning model by learning by deep learning using more than 15,000 defect images for learning, the defect classification learning model (strictly speaking, a calculation formula) is used to perform a defect classification step. Was done. As a result, there have been many defective images that can be classified accurately even though they are defective images that were difficult or impossible to be classified in the past. Representative examples of such defect images are shown in FIGS. 12 to 14.

In the defect image G4 (1) shown in FIG. 12A, the type of defect is crack Z5, and in the defect image G4 (2) shown in FIG. 12B, the type of defect is black foreign matter Z6. Since both Z5 and Z6 look black, they could not be distinguished in the past. However, in the defect classification step according to the present invention, it has become possible to accurately classify both Z5 and Z6 into different types based on the shape characteristics of the crack Z5 and the shape characteristics of the black foreign matter Z6.

In the defect image G4 (3) shown in FIG. 13 (a), the defect type is a white three-dimensional foreign matter Z7, and in the defect image G4 (4) shown in FIG. 13 (b), the defect type is white. It is a planar foreign body Z8 of the system. Since both Z7 and Z8 look white, they could not be distinguished in the past. However, in the defect classification step according to the present invention, the three-dimensional foreign matter Z7 has a shadow and the planar foreign matter Z8 has no shadow, so that both Z7 and Z8 can be accurately classified into different types. became.

In the defect image G4 (5) shown in FIG. 14 (a), the defect type is a black-based three-dimensional foreign matter Z9, and in the defect image G4 (6) shown in FIG. 14 (b), the defect type is black. It is a planar foreign body Z10 of the system. Since both Z9 and Z10 look black, they could not be distinguished in the past. However, in the defect classification step according to the present invention, the three-dimensional foreign matter Z9 is not uniformly black, and the planar foreign matter Z10 is uniformly black. Therefore, both Z9 and Z10 are accurately classified into different types. You can now do it.

If the accuracy rate of defect classification decreases due to a classification error or a sudden defect appearing while the defect classification process is repeated, the pass / fail judgment learning model shown in FIG. 8 is used. The defect classification learning model may be updated in the same manner as the update. Further, the defect classification learning model may be updated regularly. Here, the sudden defect means a defect that does not belong to any of the types of defects classified into a plurality of defects in all the defect classification steps performed before the accuracy rate starts to decrease.

FIG. 15 is a schematic configuration diagram showing the work inspection system 1 according to this embodiment in more detail. As shown in the figure, the work inspection system 1 includes the above-mentioned image pickup means 2 for imaging the work W, a work image extraction processing unit 15 for performing a work image extraction step 12 in response to a signal from the image pickup means 2, and a work. The first determination processing unit 16 that performs the first determination step 13 based on the work image extracted by the image extraction processing unit 15, and the second determination step 14 based on the work image extracted by the work image extraction processing unit 15. It is provided with a second determination processing unit 17 to perform. In this case, the work image extraction processing unit 15, the first determination processing unit 16, and the second determination processing unit 17 are all provided in the processing unit 8 shown in FIG.

Further, the work inspection system 1 includes a defect image extraction processing unit 18 that extracts a defect image from a defective work image when the determination results of the first determination processing unit 16 and the second determination processing unit 17 are defective. A defect classification processing unit 19 that performs the above-mentioned defect classification step based on the defect image extracted by the defect image extraction processing unit 18 is provided. The defect image extraction processing unit 18 and the defect classification processing unit 19 are also provided in the processing unit 8 shown in FIG.

Therefore, each operation in the work image extraction processing unit 15, the first determination processing unit 16, the second determination processing unit 17, the defect image extraction processing unit 18, and the defect classification processing unit 19 in the work inspection system 1 and their mutual operations are performed. A series of connected operations is executed, for example, by a control means (including an image processing means) such as a CPU for executing a computer program, or an electronic information processing device having a storage means such as a memory or a hard disk.

In the above embodiment, the shape of the work W having brittleness is a disk shape or a columnar shape, but it may be a quadrangular or polygonal plate shape or a columnar shape, and may have two or more through holes. It may or may not be good.

In the above embodiment, the machine learning algorithm is deep learning, but other machine learning may be used as long as it is so-called supervised learning.

Each process in the second determination step (second determination processing unit) in the above embodiment is not limited to the above-mentioned one, and other processes as long as an algorithm that does not include machine learning is used. It may be the one that performs.

In the above embodiment, the quality of the work W is determined by the first determination process 13 (first determination processing unit 16) and the second determination process 14 (second determination processing unit 17). However, the second determination step 14 (second determination processing unit 17) may be abolished, and the quality determination of the work W may be performed only by the first determination step 13 (first determination processing unit 16).

1 Work inspection system 2 Imaging means 3 Transparent substrate 11 Imaging process 12 Work image extraction process 13 First determination process 14 Second determination process 15 Work image extraction processing unit 16 First determination processing unit 17 Second determination processing unit 18 Defect image extraction Processing unit 19 Defect classification Processing unit W Work W1 Work surface W2 Work back surface G3 Work image G4 Defect image Z10 Flat foreign matter Z2 Bali Z3 Bali Z4 Black spot foreign matter Z5 Crack Z6 Black foreign matter Z7 White three-dimensional foreign matter Z8 White Planar foreign matter Z9 Black-based three-dimensional foreign matter Z10 Black-based flat foreign matter

Claims (19)

It is a work inspection method that inspects the presence or absence of defects in a work having brittleness and determines the quality of the work.
The work is imaged, the work image is extracted, and the work image is extracted.
Using the pass / fail judgment learning model constructed by machine learning, the first judgment step of inspecting the extracted work image for the presence or absence of defects and making a pass / fail judgment of the work,
It has a second determination step of inspecting the extracted work image for defects and determining the quality of the work by using an algorithm program that does not include machine learning.
The front surface and the back surface of the work are the target surfaces for defect inspection, and the work is placed on a transparent substrate and the work is imaged from the front side to obtain one work image and the work is transparent. A work inspection method characterized in that another work image is obtained by taking an image from the back side via a substrate . Claim 1 is characterized in that the first determination step is performed before the second determination step, and when the work is determined to be defective in the first determination step, the second determination step is not performed. Work inspection method described in. The first determination step is characterized in that the probability that the work is good is calculated, and the result of the quality determination of the work is obtained based on the probability and a predetermined threshold value for the probability. The work inspection method according to claim 1 or 2. One that is used to construct the pass / fail judgment learning model by placing the work on a transparent substrate and imaging the work from the front side, where the front surface and the back surface of the work are the target surfaces for defect inspection. The claim is characterized in that a learning work image is obtained and another learning work image used for constructing the pass / fail judgment learning model is obtained by imaging the work from the back side via the transparent substrate. The work inspection method according to any one of 1 to 3 . One modification process of rotating at least one of the learning work images of the one learning work image and the other learning work image obtained for use in constructing the pass / fail judgment learning model by a predetermined angle. Of the other modification processes in which at least one of the learning work image of the one learning work image and the other learning work image is inverted on both sides of a straight line passing through the central portion thereof, at least one modification process is performed. 4. The present invention is characterized in that a new learning work image is created, and the pass / fail judgment learning model is constructed by using the new learning work image and the learning work image before the modification process. Work inspection method described in. The first determination step is characterized in that at least one of a chipped edge of the work, a burr protruding in the thickness direction of the edge of the work, and a black spot foreign substance scattered and adhered to the work is recognized as the defect of the work. The work inspection method according to any one of claims 1 to 5 . The work according to claim 6 , wherein the black spot foreign matter scattered and adhered to the work recognizes whether or not the black spot foreign matter is a defect according to the size, number, and dense state of the black spot foreign matter. Inspection methods. When the result of the quality determination of the work obtained from the first determination step and the second determination step is defective, the defect in the defective work image is used by using the defect classification learning model constructed by machine learning. However, the work inspection method according to any one of claims 1 to 7 , wherein the defect classification step is performed to classify which of the types of defects is classified into a plurality of types in advance. A plurality of defective learning work images are selected from a plurality of learning work images used when constructing the pass / fail judgment learning model, and defects and their peripheral regions are selected from the defective learning work images. The work inspection method according to claim 8 , wherein a plurality of learning defect images as partial images consisting of chisel are extracted, and the defect classification learning model is constructed by using the learning defect images. .. One modification process that rotates the learning defect image obtained for use in constructing the defect classification learning model by a predetermined angle, and another that inverts the learning defect image on both sides of a straight line passing through the central portion thereof. By performing at least one of the modification processes, a new learning defect image is created, and the defect classification learning is performed using the new learning defect image and the learning defect image before the modification process. The work inspection method according to claim 9 , wherein a model is constructed. The defect classification step according to any one of claims 8 to 10 , wherein the crack generated in a part of the work and the black foreign matter adhering to the part of the work are classified as different types of defects. Work inspection method. The defect classification step is characterized in that white or black three-dimensional foreign matter adhering to a part of the work and planar foreign matter having the same color as the three-dimensional foreign matter are classified as different types of defects. The work inspection method according to any one of claims 8 to 11 . When the correct answer rate as a result of the pass / fail judgment in the first determination step decreases, any of the types of defects classified into the plurality of defects in all the defect classification steps performed before the correct answer rate starts to decrease. Any of claims 8 to 12 , wherein the pass / fail determination learning model is updated by machine learning using a defective work image having a defect that does not belong to the above type as a learning work image. Work inspection method described in Crab. The work according to any one of claims 1 to 13 , wherein the work is a tablet-shaped ceramic sintered product, and the work image is an image of the work viewed in the thickness direction. Inspection methods. The work inspection method according to any one of claims 1 to 14 , wherein the machine learning algorithm is deep learning. It is a work inspection method that inspects the presence or absence of defects in a work having brittleness and determines the quality of the work.
The work is imaged, the work image is extracted, and the work image is extracted.
Using the pass / fail judgment learning model constructed by machine learning, the first judgment step of inspecting the extracted work image for the presence or absence of defects and making a pass / fail judgment of the work,
It has a second determination step of inspecting the extracted work image for the presence or absence of defects and determining the quality of the work by using an algorithm program that does not include machine learning.
One that is used to construct the pass / fail judgment learning model by placing the work on a transparent substrate and imaging the work from the front side, where the front surface and the back surface of the work are the target surfaces for defect inspection. A work inspection characterized in that a work image for learning is obtained and another work image for learning used for constructing the pass / fail judgment learning model is obtained by imaging the work from the back side via the transparent substrate. Method. One modification process of rotating at least one of the learning work image of the one learning work image and the other learning work image obtained for use in constructing the pass / fail judgment learning model by a predetermined angle. Of the other modification processes in which at least one of the learning work image of the one learning work image and the other learning work image is inverted on both sides of a straight line passing through the central portion thereof, at least one modification process is performed. The present invention is characterized in that a new learning work image is created, and the pass / fail determination learning model is constructed by using the new learning work image and the learning work image before the modification process. Work inspection method described in. It is a work inspection method that inspects the presence or absence of defects in a work having brittleness and determines the quality of the work.
The work is imaged, the work image is extracted, and the work image is extracted.
Using the pass / fail judgment learning model constructed by machine learning, the first judgment step of inspecting the extracted work image for the presence or absence of defects and making a pass / fail judgment of the work,
It has a second determination step of inspecting the extracted work image for the presence or absence of defects and determining the quality of the work by using an algorithm program that does not include machine learning.
When the result of the quality determination of the work obtained from the first determination step and the second determination step is defective, the defect in the defective work image is used by using the defect classification learning model constructed by machine learning. However, a defect classification step is performed to classify which type of defects is classified into a plurality of types in advance.
A plurality of defective learning work images are selected from a plurality of learning work images used when constructing the pass / fail judgment learning model, and defects and their peripheral regions are selected from the defective learning work images. A plurality of learning defect images as partial images consisting of chisel are extracted, and the defect classification learning model is constructed by using the learning defect images.
One modification process that rotates the learning defect image obtained for use in constructing the defect classification learning model by a predetermined angle, and another that inverts the learning defect image on both sides of a straight line passing through the central portion thereof. By performing at least one of the modification processes, a new learning defect image is created, and the defect classification learning is performed using the new learning defect image and the learning defect image before the modification process. A work inspection method characterized by building a model. It is a work inspection method that inspects the presence or absence of defects in a work having brittleness and determines the quality of the work.
The work is imaged, the work image is extracted, and the work image is extracted.
Using the pass / fail judgment learning model constructed by machine learning, the first judgment step of inspecting the extracted work image for the presence or absence of defects and making a pass / fail judgment of the work,
It has a second determination step of inspecting the extracted work image for the presence or absence of defects and determining the quality of the work by using an algorithm program that does not include machine learning.
When the result of the quality determination of the work obtained from the first determination step and the second determination step is defective, the defect in the defective work image is used by using the defect classification learning model constructed by machine learning. However, a defect classification step is performed to classify which type of defects is classified into a plurality of types in advance.
When the correct answer rate as a result of the pass / fail judgment in the first determination step decreases, any of the types of defects classified into the plurality of defects in all the defect classification steps performed before the correct answer rate starts to decrease. A work inspection method characterized in that the pass / fail determination learning model is updated by machine learning using a defective work image having a defect that does not belong to the above type as a learning work image.

Images