JP2020085774A - Method for inspecting tube glass, method for learning, and tube glass inspection device - Google Patents

Abstract

To provide a method for inspecting a tube glass, a method for learning, and a tube glass inspection device which can more accurately determine the quality of a tube glass.SOLUTION: A primary determination unit 7 executes determination of the quality of a tube glass by the image processing algorithm P of a rule base. A secondary determination unit 13 classifies the type of the defect in the tube glass determined to be defective by the primary determination unit 7 through the identification by the learning model M. A re-determination unit 14 re-determines the quality of the tube glass according to the image processing algorithm P of the rule base by using the threshold value according to the defect classification of defect.SELECTED DRAWING: Figure 1

Description

The present invention relates to a tube glass inspecting method, a learning method and a tube glass inspecting apparatus for judging quality of tube glass.

2. Description of the Related Art Conventionally, there is known a technique of performing image processing on image information obtained by capturing an image of a test object with a camera or the like to determine quality of the test object (see Patent Documents 1 and 2). In Patent Document 2, for example, by classifying the brightness of image information into a plurality of levels, a plurality of defects having different surface brightness can be detected. Further, it is also considered that the inspection object judged to be defective in the image processing quality determination is determined again to make a final quality determination of the inspection object.

JP-A-7-318508 JP, 2006-266845, A

However, there are various types of defects that occur in the inspection object, and there is a need for further improving the accuracy of quality determination for each type of defect. In addition, the re-determination may be visually performed by the operator, which causes a problem that the precision of the re-determination varies.

It is an object of the present invention to provide a tube glass inspection method, a learning method, and a tube glass inspection device, which can improve the determination accuracy of the quality determination of the tube glass.

A tube glass inspection method for solving the above problems is a method for inspecting the quality of the tube glass, and an image capturing step of capturing an image to be inspected on which the surface of the tube glass is developed, and the image to be inspected ruled. By identifying by a base image processing algorithm, a first inspection step for inspecting the quality of the tube glass, and a tube glass determined to be defective in the first inspection step by a learning model, A second inspection step of re-inspecting the quality of the tube glass is provided.

The learning method for solving the above-mentioned problem is to perform a test image taken by unfolding the surface of the tube glass after the inspection to determine the quality of the tube glass by identifying it by a rule-based image processing algorithm. A method of a learning model used when re-inspecting the quality of the tube glass determined to be defective in the inspection, in which one learning image is converted into an image based on an image processing index. An image generation step of creating a plurality of test images for each index from one learning image by processing, and generating the learning model from machine learning using the plurality of learning images as image data. A model generation step, and in the image generation step, an index corresponding to a classification of a defective product is selected from among the plurality of indexes when creating the plurality of learning images from one learning image. Then used.

The tube glass inspection device for solving the above-mentioned problems is a configuration for inspecting the quality of the tube glass, and acquires a test image in which the surface of the tube glass is developed, and the test image is a rule-based image. The quality of the tube glass is determined by identifying a first inspection unit that inspects the quality of the tube glass by a processing algorithm and a tube glass that is determined as a defective product by the first inspection unit using a learning model. And a second inspection unit for re-inspecting the quality of.

According to the present invention, it is possible to improve the determination accuracy of the quality determination of the quality of the tube glass.

The block diagram of the tube glass inspection apparatus of one embodiment. (A), (b) is a schematic diagram of the imaging process of a tube glass. The flowchart which shows the process of a tube glass inspection method. (A) is a test image with a dent, (b) is a test image with rough skin, (c) is a test image with a chip, (d) is a test with scratches Image figure. The schematic diagram of the tube glass inspection method using learning. Explanatory drawing which shows the tendency of the kind of defect which occurs. (A) is a schematic diagram of a tube glass cutting process, and (b) is a schematic diagram of a tube glass etching process.

Hereinafter, an embodiment of a tube glass inspection method, a learning method, and a tube glass inspection apparatus will be described with reference to FIGS.
As shown in FIG. 1, in the tube glass inspection device 1, a first inspection unit 3 that first determines quality of the tube glass 2 (see FIG. 2) and a first inspection unit 3 determine that the product is defective. And a second inspection unit 4 for secondarily determining the quality of the tube glass 2. The first inspection unit 3 determines the quality of the tube glass 2 by identifying the test image Dh obtained by imaging the tube glass 2 with the rule-based image processing algorithm P. The second inspection unit 4 re-determines the tube glass 2 determined to be defective by the first inspection unit 3 through machine learning. It is preferable that the machine learning is, for example, deep learning.

The first inspection unit 3 includes an image capturing unit 5 that captures an image of the tube glass 2, an image processing unit 6 that performs image processing on the captured image data Da acquired by the image capturing unit 5, and a test image Dh acquired by the image processing unit 6. Is determined by the rule-based image processing algorithm P to determine whether the quality of the tube glass 2 is good or bad.

As shown in FIGS. 2A and 2B, the imaging unit 5 detects a backlight 8 that irradiates the tube glass 2 that is a work with light, and an image of the tube glass 2 that is illuminated by the backlight 8. And a camera 9. The camera 9 includes, for example, a line sensor 10. The imaging unit 5 captures the tube glass 2 with the line sensor 10 while rotating the tube glass 2 that is a workpiece around the tube axis La, and acquires the test image Dh in which the tube glass 2 is developed.

Returning to FIG. 1, the primary determination unit 7 processes the test image Dh by the rule-based image processing algorithm P to make a primary determination of the quality of the tube glass 2. In the pass/fail judgment by the rule-based image processing algorithm P, the parameters (for example, the fineness of the line of the defect, the area, the number, the connection of the defects, etc.) of the defect image of the test image Dh appearing on the image are compared with the threshold value R. It has been judged. Therefore, if the parameter of the defective portion is less than the threshold value R, the tube glass 2 is determined to be a good product, and if the parameter of the defective portion is equal to or greater than the threshold value R, the tube glass 2 is determined to be a defective product. The primary determination unit 7 outputs the inspection image Dh of the tube glass 2 determined to be a defective product, that is, the re-inspection inspection image Dh1 to the second inspection unit 4.

The second inspection unit 4 is extracted by the defect extraction unit 11 that extracts a portion in which a defect is shown in the re-inspection inspection image Dh1 (inspection image Dh) acquired from the first inspection unit 3, and the defect extraction unit 11. And a defect cutout portion 12 for cutting out a defective portion. The second inspection unit 4 classifies the defects by identifying the image of the defect portion cut out by the defect cutting unit 12 by the learning model M, and the pipe determined to be defective by the primary determination. A re-determination unit 14 that re-determines the quality of the glass 2 using the classification result of the secondary determination unit 13 is provided.

The secondary determination unit 13 classifies the defect type of the defective product through the learning model M for the tube glass 2 determined as the defective product in the inspection process by the first inspection unit 3. The types of defects in this example are, for example, dents, rough skin, chips, and scratches. The learning model M of this example is generated by machine learning (in this example, deep learning: deep learning) an image group of the defective tube glass 2.

The re-determination unit 14 sets a threshold R to be used in the rule-based image processing algorithm P for each defect type for the re-inspection inspection image Dh1 of the defective product (tube glass 2) in which the defects are classified, By identifying defective products of the tube glass 2 with the threshold value R, the quality of the tube glass 2 is re-inspected. In the case of the present example, the threshold value R is set to the magnitude of the threshold value R, for example, according to the degree of importance of the defect. The threshold value R in this example is set high for skin roughness determination and low for determination of scratches and chips.

Next, the operation and effect of the tube glass inspection device 1 of the present embodiment will be described with reference to FIGS. 3 to 7.
In step 101, the imaging unit 5 of the first inspection unit 3 captures an image of the tube glass 2 and outputs the captured data Da to the image processing unit 6 (imaging step).

In step 102, the image processing unit 6 of the first inspection unit 3 obtains a test image Dh in which the image of the tube glass 2 is developed in the direction orthogonal to the tube axis La by image-processing the imaged data Da (imaging step). ).

In step 103, the primary determination unit 7 of the first inspection unit 3 identifies the inspection image Dh extracted by the image processing unit 6 by the rule-based image processing algorithm P, thereby determining the quality of the tube glass 2. A primary judgment is carried out to judge pass/fail (first inspection step). In the case of this example, the primary determination unit 7 performs determination based on the threshold value R using the test image Dh according to the rule-based image processing algorithm P, and confirms the quality of the tube glass 2. The threshold value R used in the first determination is set to a strict value so that any defect type can be dealt with.

In step 104, the primary determination unit 7 determines whether or not the primary determination has been established. If the primary determination is established, the process proceeds to step 105, and if the primary determination is not established, the process proceeds to step 106. When the primary determination is not established, the primary determination unit 7 outputs the inspection image Dh of the tube glass 2 for which the primary determination is not established, that is, the re-inspection inspection image Dh1 to the second inspection unit 4. ..

In step 105, the primary determination unit 7 determines that the tube glass 2 inspected at this time is “non-defective”. As a result, the tube glass 2 under inspection is transported as a product that can be shipped.

In step 106, the defect extraction unit 11 of the second inspection unit 4 extracts, from the re-inspection inspected image Dh1 acquired from the first inspection unit 3, the defect portion reflected in the image.
In step 107, the defect cutout unit 12 of the second inspection unit 4 cuts out a defect image Dh1′ from the re-inspection inspection image Dh1 in which the defective portion is extracted. By cutting out the defect image Dh1′ from the re-inspection inspection image Dh1, an enlarged image of the defect portion and its vicinity can be obtained.

In step 108, the secondary determination unit 13 of the second inspection unit 4 classifies the defects in the defect image Dh1' by identifying the defect image Dh1' input from the defect cutout unit 12 with the learning model M. Classification by the learning model M is deep learning, which is a type of machine learning.

FIG. 4 illustrates types of defects that can occur in the tube glass 2. The defects include “dents” shown in FIG. 4A, “rough skin” shown in FIG. 4B, “chips” shown in FIG. 4C, and “scratches” shown in FIG. 4D. .. The dents are large defects that are formed on the surface of the tube glass 2, and it is not good if there is even one. Rough skin is likely to occur on the etched surface of the end surface of the tube glass 2. A chip is a part where the tube glass 2 is partially lost, and is easily generated on the cut surface of the tube glass 2. The scratches are like elongated grooves on the surface of the tube glass 2. The secondary determination unit 13 can determine the type of defect occurring in one tube glass 2 and the number of each defect. Further, as another defect, there is “dirt” although not shown.

Returning to FIG. 3, in step 109, the re-determination unit 14 is determined to be defective by the primary determination using the rule-based image processing algorithm P based on the defect classification result by the secondary determination unit 13. The quality of the tube glass 2 is determined again. At this time, when performing the re-determination using the rule-based image processing algorithm P, the threshold R corresponding to the defect type is appropriately selected and the re-determination is performed.

In step 110, the re-determination unit 14 determines whether or not the re-determination has been established in the rule-based image processing algorithm P in which the threshold value R is set according to the defect type. If the redetermination is established, the process proceeds to step 105. As a result, the tube glass 2 that is determined to be a defective product in the primary determination is finally determined to be a “good product”. Therefore, in determining the quality of the tube glass 2 of a quality that does not cause a problem even when shipped, it is possible to obtain a determination result of a non-defective product from a product that has been determined as a defective product in the determination so far.

On the other hand, if the redetermination is not established, the process proceeds to step 111. As a result, the tube glass 2 is finally determined to be a “defective product”. Therefore, it becomes possible to prevent the tube glass 2 having a large defect in which both the primary determination and the secondary determination are not established from being shipped to the market.

FIG. 5 illustrates a method of using machine learning (deep learning) used in the tube glass inspection method. Machine learning has a “learning phase” and an “inference phase”. In the learning phase, learning images Dk are collected and classified, and these are learned by machine learning (deep learning) to construct an optimum learning model M. Then, in the inference phase, the learning model M generated in the learning phase is used to classify the defective portions in the inspection image Dh (re-inspection inspection image Dh1), and the classification result is used to perform rule-based image processing. The quality determination by the algorithm P is performed again to determine the quality of the tube glass 2.

The learning model M is generated by machine-learning a plurality of learning images Dk, but in order to secure the number of learning images Dk, the defect image Dh1′, which is the learning image Dk, is set to various indices F. The number of learning sheets may be increased by changing with. The various indexes F include, for example, vertical inversion, horizontal inversion, rotation, brightness change, contrast change, angle change, and position shift. However, due to the manufacturing process of the tube glass 2, each defect has a characteristic in the occurrence position and the like, and the learning image Dk obtained based on all the indexes F is effective in determining any defect. It doesn't mean there is.

For example, as shown in FIG. 6, in the developed view of the tube glass 2, the sides X1 and X2 along the tube axis La direction are boundaries of the line sensor 10, so that no unevenness occurs. On the other hand, in the development view of the tube glass 2, the sides Y1 and Y2 at the ends in the tube axis La direction are the end faces of the work, and therefore, irregularities may occur. As described above, it can be seen that the respective sides X1, X2, Y1 and Y2 in the developed view of the tube glass 2 are also characterized by unevenness.

Further, as shown in FIGS. 7A and 7B, the manufacturing process of the tube glass 2 includes a cutting step of cutting the tube glass 2 from the continuously formed base material glass 15 by the cutting device 16 (see FIG. )) and an etching step (FIG. 7B) of etching one end 17b of the tube glass 2 after cutting. Therefore, the tube glass 2 has one end 17a as a cut surface and the other end 17b as an etching surface. In the cutting step, for example, a method of forming a scribe on the base material glass 15 and then applying a stress to break it, a cutting method using a rotary blade, or a laser for cutting the base material glass 15 at a predetermined position by laser light is used. It is preferable to use a cutting method or the like. When stress is applied to the base glass material 15, stress may be applied by heating and quenching, or stress may be applied by mechanical bending.

Returning to FIG. 6, in the test image Dh in which the tube glass 2 is developed, only “chips” are generated on the cut surface due to the characteristics of mechanical processing such as cutting, and the etching surface has a chemical change due to etching processing. Due to the effect of, only "rough skin" tends to occur. As described above, in the development view of the tube glass 2, the type of defect that occurs is determined according to the type of processing of the tube glass 2. Therefore, when increasing the learning images Dk when creating the learning model M, it is effective to process the images according to the type of the index F.

For example, in the case where the defect appearing on the image is “rough skin” or “chip” that occurs only at the end of the tube glass 2, for example, learning is performed by the index F such as “upside-down”, “brightness change”, “contrast change”. The number of images Dk for use is increased. That is, the index F is selected by excluding “horizontal reversal” and “rotation”. On the other hand, when the defect appearing on the image is a “dent” or a “scratch”, learning is performed by the index F such as “vertical flip”, “horizontal flip”, “rotation”, “luminance change”, “contrast change”, etc. The number of images Dk for use is increased. In this way, the learning model M can be optimized and the recognition ability of defect classification can be improved.

Further, since the dent is the least preferable as a defect, the tube glass 2 may be determined as a defective product at the time when the dent is classified as the dent in the secondary determination. Since it is not preferable that a plurality of rough skins are detected, the tube glass 2 may be determined as a defective product when a plurality of rough skins are detected in the secondary determination.

As described above, according to the present example, the tube glass 2 determined to be defective in the first inspection process by the rule-based image processing algorithm P is re-determined in the second inspection process using the learning model M. At the time of re-determination, if it is determined that there is no problem in the second inspection step, the tube glass 2 is treated as a good product. Therefore, it is not necessary to judge the tube glass 2 which is actually a good product even if it is handled as a non-defective product, as a defective product. Therefore, the determination accuracy of the quality determination of the tube glass 2 can be improved.

In the second inspection process, for the tube glass 2 determined to be defective in the first inspection step, the defect type of the defective product is classified through identification by the learning model M, and the classification result is used to determine the quality of the tube glass 2. Re-examine the quality. As a result, it is possible to re-determine the quality of the tube glass 2 in an optimum manner according to the result of classifying the defects. Therefore, it is possible to further contribute to ensuring the quality determination accuracy of the quality of the tube glass 2.

In the second inspection step, the tube glass 2 in which the defects are classified is identified by the image processing algorithm P in which the threshold value R is set for each type of defect, and the quality of the tube glass 2 is re-inspected. To do. Therefore, even when the quality of the tube glass 2 is determined to be good or bad by the rule-based image processing algorithm P at the time of re-determination, high determination accuracy can be ensured.

The type of defect is at least one of dents, rough skin, chips, scratches and stains. Therefore, it is possible to perform the re-determination in an optimum manner according to each defect such as a dent, rough skin, chip, scratch, and stain that can occur on the tube glass 2.

The learning model M is generated by machine learning using a test image Dh of the defective tube glass 2 as a learning image Dk and using the learning image Dk as image data. Therefore, the learning model M is generated by learning the actual test image Dh as the learning image Dk, so that the learning model M with high determination accuracy can be created.

The learning model M is generated by machine learning using a test image Dh of the defective tube glass 2 as a learning image Dk and using the learning image Dk as image data. Therefore, the quality determination of the tube glass 2 is performed by the learning model M that has learned the actual inspected image Dh of the inspection image Dh, and thus the learning model M with high determination accuracy can be created.

By processing one learning image Dk on the basis of the image processing index F, a learning image Dk is generated for each index F from one learning image Dk, and a plurality of these learning images Dk are generated. A learning model M is generated from the machine learning using the image data as. Therefore, the learning model M can be generated from a large number of learning images Dk, which is advantageous for generating the learning model M with good determination accuracy.

When a plurality of learning images Dk are created from one learning image Dk through each index F, the index F corresponding to the defect classification is selected and used from among the plurality of indexes F. Therefore, when creating a plurality of learning images Dk from one learning image Dk, the learning images Dk are increased by selectively using the index F corresponding to each type of defect. Therefore, the learning image Dk used to create the learning model M can be optimized, which is even more advantageous for generating a highly accurate learning model M.

The image processing index F is any one of vertical inversion, horizontal inversion, rotation, brightness change, contrast change, angle change, position shift, or a combination thereof. Therefore, it is possible to process one image with the suitable index F and increase the number of learning images Dk.

When the classification of defects imaged in one learning image Dk to be image-processed is a defect that occurs only at the end of the tube glass 2, the left-right inversion and rotation are excluded from the index F of the image processing. The index F is selected. Therefore, the number of learning images Dk can be increased based on the appropriate index F according to the type of defect.

In the imaging step, the tube glass 2 is arranged between the backlight 8 and the camera 9 (line sensor 10), and the tube glass 2 is rotated around the tube axis La and an image is taken by the camera 9 (line sensor 10). , The image Dh to be inspected is acquired. Therefore, an image in which the tube glass 2 is developed can be easily acquired by performing image-capturing photography while rotating the tube glass 2 around the tube axis La.

The manufacturing process of the tube glass 2 includes a cutting step of cutting the tube glass 2 from the base material glass 15 and an etching step of etching one end of the tube glass 2 after cutting. When manufacturing the tube glass 2, the end 17b of the tube glass 2 is etched, so that the end 17b of the tube glass 2 can be surface-finished. In the imaging step, the test image Dh is captured so that at least one of the etching surface formed on the one end 17b of the tube glass 2 and the cut surface formed on the other end 17a is reflected. Preferably. By doing so, even if a defect occurs on the end face during cutting or etching, it can be classified and extracted by the secondary determination. Note that if the tube glass 2 is transparent, it is easy to capture the test image Dh so that both end surfaces are reflected at the same time.

Further, in the tube glass inspection method, the inspected image Dh of the tube glass 2 determined to be defective in the first inspection step and the second inspection step is acquired as learning data, and the learning model M is learned based on the learning data. A learning process to be executed may be provided. In this case, the learning model M can be sequentially updated by learning the image data of the tube glass 2 which is determined to be a defective product in the inspection process, so that the learning model M can be optimized. Therefore, it is more advantageous to secure high determination accuracy.

The present embodiment can be modified and implemented as follows. The present embodiment and the following modified examples can be implemented in combination with each other within a technically consistent range.
In the secondary judgment, when “reliability” is required as one parameter of the classification result, this reliability may be used for the defect classification judgment. For example, the reliability of the classification itself is improved by validating the classification result with high reliability and invalidating the classification result with low reliability. In this case, the accuracy of the quality judgment of the quality of the tube glass 2 is further improved.

The rule-based image processing algorithm P can be of various formats and methods as long as it is a rule-based program.
The re-determination is not limited to the process using the rule-based image processing algorithm P, and may be changed to another process such as a process using learning.

The re-determination process may be performed by the primary determination unit 7, for example.
-The re-judgment may include visual inspections performed by the inspector during the implementation.
The learning model M is not limited to being generated by deep learning (deep learning), and various machine learning methods can be applied.

The type of classification may include types other than the examples.
As the image processing index F, various parameters such as image size and image saturation can be used as long as they are parameters that change the image.

The update of the learning model M may be omitted.
The method of manufacturing the tube glass 2 may employ steps other than the cutting step and the etching step described in the embodiments. Further, the tube glass 2 may be created through any manufacturing process.

The tube glass 2 can be used as a glass material in various fields such as medical use and lighting. In addition, the tube glass 2 can be applied in various sizes from a fine one to a certain size.

DESCRIPTION OF SYMBOLS 1... Tube glass inspection device, 2... Tube glass, 3... 1st inspection part, 4... 2nd inspection part, 8... Backlight, 9... Camera, 10... Line sensor, 15... Base glass, 17a, 17b... Edge, Dh... Test image, Dk... Learning image, P... Image processing algorithm, M... Learning model, F... Index, R... Threshold, La... Pipe axis.

Claims (12)

A tube glass inspection method for inspecting the quality of tube glass,
An image pickup step of picking up an image to be inspected by unfolding the surface of the tube glass;
A first inspection step for inspecting the quality of the tube glass by identifying the image to be inspected by a rule-based image processing algorithm;
A tube glass inspection method comprising: a second inspection step of re-inspecting the quality of the tube glass by identifying a tube glass determined to be defective in the first inspection step with a learning model. In the second inspection step, with respect to the tube glass determined to be defective in the first inspection step, the defect type of the defective article is classified through the identification by the learning model, and the quality of the tube glass is classified using the classification result. The tube glass inspection method according to claim 1, wherein the quality of the tube glass is reinspected. In the second inspection step, the quality of the tube glass is re-inspected by identifying the tube glass in which defects are classified by the image processing algorithm in which a threshold value is set for each type of defect. The tube glass inspection method according to claim 2. The tube glass inspection method according to claim 2 or 3, wherein the type of the defect is at least one of a dent, a rough skin, a chip, a scratch, and a stain. The learning model is generated by machine learning using a test image of the defective tube glass as a learning image,
The learning image is modified and duplicated by processing one learning image based on an image processing index,
The tube according to claim 4, wherein a plurality of types of the index for image processing are prepared, and an index is selected and used according to a classification of defects captured in one learning image that is a target for the image processing. Glass inspection method. The tube glass inspection method according to claim 5, wherein the image processing index is any one of vertical inversion, horizontal inversion, rotation, brightness change, contrast change, angle change, position shift, or a combination thereof. When the classification of defects imaged in the one learning image to be image-processed is a defect that occurs only at the end of the tube glass, left-right inversion and rotation are excluded from the image-processing index. The tube glass inspection method according to claim 6, wherein the index is selected by the following method. In the image capturing step, the tube glass is arranged between a backlight and a camera, and the test image is acquired by capturing an image with the camera while rotating the tube glass around a tube axis. 7. The tube glass inspection method according to any one of 7. A cutting step of cutting the tube glass from the continuously formed base material glass,
After cutting, comprising an etching step of etching one end of the tube glass,
In the imaging step, the test image is captured so that at least one of an etching surface formed at one end of the tube glass and a cut surface formed at the other end of the tube glass is reflected. The tube glass inspection method as described in any one of 8-8. A learning step of acquiring, as learning data, an inspected image of the tube glass determined to be the defective product in the first inspection step and the second inspection step, and executing learning of the learning model based on the learning data. The tube glass inspection method according to any one of claims 1 to 9, further comprising: The test image taken by unfolding the surface of the tube glass was inspected to determine the quality of the tube glass by identifying it by a rule-based image processing algorithm, and was then determined to be defective in the inspection. A learning method of a learning model used when reinspecting the quality of the tube glass,
An image generating step of processing a single learning image based on an image processing index to create a plurality of test images for each index from the single learning image,
A model generating step of generating the learning model from machine learning using a plurality of learning images as image data,
In the image generation step, in creating a plurality of learning images from one learning image, a learning method is used in which an index corresponding to a defective product classification is selected from the plurality of indexes. . A tube glass inspection device for inspecting the quality of tube glass,
A first inspection unit for inspecting the quality of the tube glass by obtaining an image to be inspected by unfolding the surface of the tube glass and identifying the image to be inspected by a rule-based image processing algorithm,
A tube glass inspecting apparatus, comprising: a second inspecting section that re-inspects whether the quality of the tube glass is good by identifying a tube glass determined to be defective by the first inspecting section by a learning model.