JP7151426B2 - Tube glass inspection method, learning method and tube glass inspection device - Google Patents

Description

TECHNICAL FIELD The present invention relates to a tube glass inspection method, a learning method, and a tube glass inspection apparatus for determining quality of tube glass.

2. Description of the Related Art Conventionally, there is a well-known technique for determining quality of an inspection object by image processing image information obtained by imaging the inspection object with a camera or the like (see Patent Documents 1 and 2, etc.). In Patent Document 2, for example, by classifying the brightness of image information into a plurality of levels, it is possible to detect a plurality of defects having different surface brightnesses. It is also being considered to re-determine an inspection object that has been determined to be defective in the image processing pass/fail determination to finally make a pass/fail determination of the inspection object.

JP-A-7-318508 JP 2006-266845 A

However, there are various types of defects that occur in inspection objects, and there is a need for further improvement in the accuracy of quality determination for various defects. In addition, the re-determination may be performed visually by the operator, and there is also a problem that the accuracy of the re-determination varies.

SUMMARY OF THE INVENTION An object of the present invention is to provide a glass tube inspection method, a learning method, and a glass tube inspection apparatus capable of improving the accuracy of determining the quality of a glass tube.

A tube glass inspection method for solving the above-mentioned problems is a method for inspecting the quality of a tube glass, comprising: an imaging step of taking an inspection image of the surface of the tube glass developed; A first inspection step of inspecting the quality of the glass tube by identifying it with a base image processing algorithm; and a second inspection step of reinspecting the quality of the tube glass.

A learning method for solving the above-mentioned problem is to discriminate, by a rule-based image processing algorithm, an image to be inspected obtained by unfolding the surface of the glass tube. , a learning model method used when re-inspecting the quality of the tube glass determined to be defective in the inspection, wherein one learning image is processed based on an image processing index. an image generating step of generating a plurality of test images for each index from the single learning image by processing; and generating the learning model from machine learning using the plurality of learning images as image data. and a model generation step, wherein in the image generation step, an index corresponding to the classification of defective products is selected from among the plurality of indices when generating a plurality of the learning images from the single learning image. used as

A tube glass inspection apparatus for solving the above-mentioned problems is configured to inspect quality of a tube glass, acquires an inspection image of the surface of the tube glass developed, and transforms the inspection image into a rule-based image. A first inspection unit that inspects the quality of the glass tube by identifying it using a processing algorithm, and a glass tube that is determined to be defective by the first inspection unit by identifying the quality of the glass tube by using a learning model. and a second inspection unit for re-inspecting whether the quality is good or bad.

ADVANTAGE OF THE INVENTION According to this invention, the judgment precision of quality judgment of the quality of a tube glass can be improved.

1 is a configuration diagram of a tube glass inspection apparatus according to one embodiment; FIG. (a), (b) is a schematic diagram of the imaging process of a tube glass. The flowchart which shows the process of the tube glass inspection method. (a) is a test image showing dents, (b) is a test image showing rough skin, (c) is a test image showing chips, and (d) is a test image showing scratches image diagram. Schematic diagram of tube glass inspection method using learning. Explanatory drawing which shows the tendency of the defect type which generate|occur|produces. (a) is a schematic diagram of the cutting process of the tube glass, and (b) is a schematic diagram of the etching process of the tube glass.

An embodiment of a tube glass inspection method, a learning method, and a tube glass inspection apparatus will be described below with reference to FIGS. 1 to 7. FIG.
As shown in FIG. 1, the glass tube inspection apparatus 1 includes a first inspection unit 3 that primarily determines quality of a glass tube 2 (see FIG. 2, etc.), and a product that is determined to be defective by the first inspection unit 3. and a second inspection unit 4 for secondarily judging quality of the tube glass 2 . The first inspection unit 3 determines the quality of the glass tube 2 by identifying the inspection image Dh obtained by imaging the glass tube 2 with a rule-based image processing algorithm P. As shown in FIG. The second inspection unit 4 re-determines the glass tube 2 determined as defective by the first inspection unit 3 through machine learning. Machine learning is preferably deep learning, for example.

The first inspection unit 3 includes an imaging unit 5 that photographs the tube glass 2, an image processing unit 6 that performs image processing on the imaging data Da acquired by the imaging unit 5, and an inspection image Dh acquired by the image processing unit 6. is identified by a rule-based image processing algorithm P to determine whether the quality of the tube glass 2 is good or bad.

As shown in FIGS. 2(a) and 2(b), the imaging unit 5 detects a backlight 8 that irradiates light onto the glass tube 2, which is a work, and an image of the glass tube 2 illuminated by the backlight 8. A camera 9 is provided. The camera 9 consists of a line sensor 10, for example. The imaging unit 5 captures an image of the glass tube 2, which is a workpiece, by the line sensor 10 while rotating the glass tube 2 around the tube axis La, thereby acquiring an image Dh to be inspected in which the glass tube 2 is developed.

Returning to FIG. 1 , the primary determination unit 7 processes the test image Dh with a rule-based image processing algorithm P to primarily determine whether the quality of the tube glass 2 is good or bad. The quality judgment by the rule-based image processing algorithm P compares the parameters of the defective portion of the inspection image Dh (for example, the line width, area, number of defects, connection of defects, etc.) with the threshold value R. It has been judged. Therefore, if the parameter of the defective portion is less than the threshold value R, the glass tube 2 is determined to be non-defective, and if the parameter of the defective portion is equal to or greater than the threshold value R, the glass tube 2 is determined to be defective. The primary determination unit 7 outputs to the second inspection unit 4 a test image Dh of the tube glass 2 determined to be defective, that is, a re-inspection test image Dh1.

The second inspection unit 4 includes a defect extraction unit 11 for extracting a portion where a defect appears in the re-inspection image Dh1 (inspection image Dh) acquired from the first inspection unit 3, and a defect extracted by the defect extraction unit 11. and a defect extraction unit 12 for extracting the defective portion. The second inspection unit 4 includes a secondary determination unit 13 that classifies defects by identifying images of defective locations cut out by the defect cutout unit 12 using a learning model M, and a tube that is determined to be defective in the primary determination. A re-determination unit 14 for re-determining the quality of the glass 2 using the classification result of the secondary determination unit 13 is provided.

The secondary determination unit 13 classifies the type of defect of the defective product through identification by the learning model M for the tube glass 2 determined as defective in the inspection process by the first inspection unit 3 . The types of defects in this example are, for example, dents, rough skin, chips, and scratches. The learning model M in this example is generated by machine learning (in this example, deep learning) an image group of the glass tube 2 which is a defective product.

The re-determining unit 14 sets a threshold value R to be used in a rule-based image processing algorithm P for each type of defect for a re-inspected image Dh1 of a defective product (glass tube 2) classified into defects, By identifying the defective glass tube 2 with the threshold R, the quality of the glass tube 2 is reinspected. In the case of this example, the threshold R is set to be large or small according to the degree of importance of the defect, for example. The threshold value R in this example is set high for judging rough skin, for example, and is set low for judging scratches, chips, and the like.

Next, the action and effect of the tube glass inspection apparatus 1 of this embodiment will be described with reference to FIGS. 3 to 7. FIG.
In step 101, the imaging section 5 of the first inspection section 3 takes an image of the tube glass 2 and outputs the imaging data Da to the image processing section 6 (imaging step).

In step 102, the image processing unit 6 of the first inspection unit 3 performs image processing on the imaging data Da to obtain an inspection image Dh obtained by developing the image of the tube glass 2 in the direction perpendicular to the tube axis La (imaging step ).

In step 103, the primary determination unit 7 of the first inspection unit 3 determines the quality of the tube glass 2 by identifying the test image Dh extracted by the image processing unit 6 using the rule-based image processing algorithm P. A primary judgment for judging quality is executed (first inspection step). In this example, the primary determination unit 7 uses the test image Dh according to the rule-based image processing algorithm P to perform determination based on the threshold value R, and confirms the quality of the tube glass 2 . The threshold value R used in the first determination is set to a rather strict value so as to be able to deal with any defect type.

At step 104, the primary determination unit 7 determines whether or not the primary determination is established. If the primary determination is established, the process proceeds to step 105, and if the primary determination is not established, the process proceeds to step 106. When the primary determination is not established, the primary determination unit 7 outputs the inspection image Dh of the tube glass 2 for which the primary determination was not established, that is, the re-inspection image Dh1 to the second inspection unit 4. .

At step 105, the primary determination unit 7 determines that the glass tube 2 inspected at this time is a "non-defective product". As a result, the glass tube 2 currently being inspected is transported as a product that can be shipped.

At step 106 , the defect extracting section 11 of the second inspection section 4 extracts the defect portion appearing in the image from the re-inspection image Dh<b>1 acquired from the first inspection section 3 .
In step 107, the defect clipping section 12 of the second inspection section 4 clips a defect image Dh1' from the re-inspection image Dh1 in which the defect location has been extracted. By cutting out the defect image Dh1′ from the re-inspection image Dh1, an enlarged image of the defect location and its vicinity can be obtained.

In step 108, the secondary determination unit 13 of the second inspection unit 4 identifies the defect image Dh1' input from the defect extraction unit 12 using the learning model M, thereby classifying the defect appearing in the defect image Dh1'. Classification by the learning model M is deep learning, which is a type of machine learning.

FIG. 4 illustrates the types of defects that can occur in the tube glass 2 . Defects include "dents" shown in FIG. 4(a), "rough skin" shown in FIG. 4(b), "chipping" shown in FIG. 4(c), and "scratches" shown in FIG. 4(d). . A dent is a large defect formed on the surface of the tube glass 2, and even one dent is not good. Rough skin tends to occur on the etched surface of the end surface of the tube glass 2 . A chip is a part of the glass tube 2 that is partially chipped, and is likely to occur on the cut surface of the glass tube 2 . A scratch is like an elongated groove on the surface of the tube glass 2 . The secondary determination unit 13 can determine the types of defects occurring in one glass tube 2 and the number of defects. In addition, there is also "dirt" as another defect (not shown).

Returning to FIG. 3, in step 109, the re-determining unit 14 uses the rule-based image processing algorithm P based on the defect classification result by the secondary determining unit 13 to determine that the defect is defective in the primary determination. The quality of the tube glass 2 is determined again. At this time, in performing re-determination using the rule-based image processing algorithm P, the threshold value R corresponding to the defect type is appropriately selected and re-determination is performed.

In step 110, the re-determination unit 14 determines whether or not the re-determination by the rule-based image processing algorithm P in which the threshold value R is set according to the defect type is established. If the re-determination is established, the process proceeds to step 105 . As a result, the glass tube 2, which was determined as a defective product in the primary determination, is finally determined as a "non-defective product". Therefore, when judging the quality of the tube glass 2 which is of a quality that can be shipped without any problem, it is possible to obtain the result of judging that the tube glass 2, which has been judged as a defective product in the past judgment, is a non-defective product.

On the other hand, if the re-determination is not established, the process proceeds to step 111 . As a result, the glass tube 2 is finally determined to be "defective". Therefore, the glass tube 2 having a large defect for which both the primary determination and the secondary determination are not satisfied can be prevented from being shipped to the market.

FIG. 5 illustrates the use of machine learning (deep learning) for tube glass inspection methods. Machine learning has a “learning phase” and an “inference phase”. In the learning phase, learning images Dk are collected and classified, and these are learned by machine learning (deep learning) to construct an optimal learning model M. FIG. Then, in the inference phase, the learning model M generated in the learning phase is used to classify the defect locations appearing in the test image Dh (re-inspection test image Dh1), and the classification results are used to perform rule-based image processing. The quality determination by the algorithm P is performed again, and the quality of the glass tube 2 is determined.

The learning model M is generated by machine-learning a plurality of learning images Dk. , and the number of learning sheets may be increased. Various indices F include, for example, vertical inversion, horizontal inversion, rotation, brightness change, contrast change, angle change, position shift, and the like. However, due to the manufacturing process of the tube glass 2, various defects have characteristics such as their occurrence positions, etc., and the learning image Dk obtained based on all the indices F is effective in determining any defect. It doesn't mean that there is.

For example, as shown in FIG. 6, in the developed view of the glass tube 2, the sides X1 and X2 along the direction of the tube axis La are the boundaries of the line sensor 10, so that unevenness does not occur. On the other hand, in the developed view of the glass tube 2, the sides Y1 and Y2 at the ends in the direction of the tube axis La are workpiece end faces, and therefore there is a possibility that unevenness will occur. As described above, it can be seen that each side X1, X2, Y1, Y2 in the development view of the glass tube 2 also has a feature in the generation of unevenness.

Further, as shown in FIGS. 7A and 7B, the manufacturing process of the glass tube 2 is a cutting process in which the glass tube 2 is cut by a cutting device 16 from a continuously molded base glass 15 (see FIG. 7A). )) and an etching step (FIG. 7(b)) for etching one end 17b of the tube glass 2 after cutting. Therefore, the glass tube 2 has one end 17a as a cut surface and the other end 17b as an etched surface. The cutting process includes, for example, a method of forming a scribe on the base glass 15 and then applying stress to break it, a cutting method using a rotary blade, and a laser cutting method that cuts the base glass 15 at a predetermined position with a laser beam. It is preferable to use a cutting method or the like. When stress is applied to the base material glass 15, the stress may be applied by heating and rapid cooling, or may be applied by mechanical bending.

Returning to FIG. 6, in the test image Dh in which the tube glass 2 is developed, only "chipping" occurs on the cut surface due to the mechanical processing characteristics of cutting, and on the etching surface, there is no chemical change due to the etching process. Due to the influence of , only "rough skin" tends to occur. As described above, in the developed view of the glass tube 2 , the type of defect that occurs is determined according to the type of processing of the glass tube 2 . Therefore, when increasing the learning images Dk when creating the learning model M, it is effective to process the images according to the type of the index F.

For example, in the case of a "rough surface" or "chipping" that occurs only at the end of the glass tube 2, the defects appearing on the image are learned using an index F such as "vertical inversion", "brightness change", and "contrast change". The number of images Dk for use is increased. That is, the index F is selected by excluding "horizontal reversal" and "rotation". On the other hand, if the defects appearing on the image are "dents" or "scratches", learning is performed using indices F such as "vertical inversion", "horizontal inversion", "rotation", "brightness change", and "contrast change". The number of images Dk for use is increased. In this way, it is possible to optimize the learning model M and improve the recognizability of defect classification.

Also, since dents are the most undesirable defects, the glass tube 2 may be determined as defective when classified as dents in the secondary determination. Since it is not preferable to detect a plurality of rough skins, the tube glass 2 may be judged as defective when a plurality of rough skins are detected in the secondary judgment.

As described above, according to this example, the glass tube 2 determined as defective in the first inspection process by the rule-based image processing algorithm P is re-determined in the second inspection process using the learning model M. At the time of re-judgment, if it is judged that there is no problem in the second inspection process, this glass tube 2 is treated as a non-defective product. Therefore, it is possible to avoid judging the glass tube 2 as a defective product when it can be treated as a non-defective product. Therefore, it is possible to improve the accuracy of determining whether the quality of the tube glass 2 is good or bad.

In the second inspection process, the glass tube 2 determined to be defective in the first inspection process is classified by the type of defect of the defective product through identification by the learning model M, and the quality of the glass tube 2 is evaluated using the classification result. Re-inspect for good or bad. As a result, it is possible to re-determine whether the quality of the glass tube 2 is good or bad in an optimum manner according to the defect classification result. Therefore, it is possible to further contribute to ensuring the quality determination accuracy of the tube glass 2 .

In the second inspection step, the quality of the glass tube 2 is reinspected by identifying the glass tube 2 classified with defects by an image processing algorithm P in which a threshold value R is set for each type of defect. do. Therefore, even if the rule-based image processing algorithm P is used to determine whether the quality of the tube glass 2 is good or bad at the time of re-determination, high determination accuracy can be ensured.

The defect type is at least one of dents, rough skin, chips, scratches, and stains. Therefore, re-determination can be carried out in an optimum manner according to defects such as dents, rough skin, chips, scratches, and stains that may occur on the tube glass 2 .

The learning model M is generated by machine learning using the test image Dh of the defective tube glass 2 as the learning image Dk and using the learning image Dk as image data. Therefore, the learning model M is generated by learning the actual test image Dh as the learning image Dk, so that the learning model M with high determination accuracy can be created.

The learning model M is generated by machine learning using the test image Dh of the defective tube glass 2 as the learning image Dk and using the learning image Dk as image data. Therefore, the quality determination of the tube glass 2 is performed by the learning model M that has learned the test image Dh of the actual defective product, so that the learning model M with high determination accuracy can be created.

By processing one learning image Dk based on the image processing index F, a learning image Dk is generated for each index F from one learning image Dk, and these plural learning images Dk are generated. A learning model M is generated from machine learning using M as image data. Therefore, it is possible to generate the learning model M from a large number of learning images Dk, which is advantageous for generating the learning model M with high judgment accuracy.

When a plurality of learning images Dk are created from one learning image Dk through each index F, the index F corresponding to the defect classification is selected and used from among the plurality of indices F. Therefore, when creating a plurality of learning images Dk from one learning image Dk, the number of learning images Dk is increased by selectively using an index F corresponding to each defect type. Therefore, the learning images Dk used for creating the learning model M can be optimized, which is more advantageous for generating the learning model M with high accuracy.

The image processing index F is any one of vertical inversion, horizontal inversion, rotation, brightness change, contrast change, angle change, position shift, or a combination thereof. Therefore, it is possible to increase the number of learning images Dk by processing one image using a suitable index F.

If the defect imaged in one learning image Dk to be processed is classified as defects occurring only at the end of the glass tube 2, left-right reversal and rotation are excluded from the index F for image processing. Index F is selected. Therefore, the number of learning images Dk can be increased based on an appropriate index F according to the type of defect.

In the imaging process, the glass tube 2 is placed between the backlight 8 and the camera 9 (line sensor 10), and the camera 9 (line sensor 10) takes an image while rotating the glass tube 2 around the tube axis La. , to acquire the test image Dh. Therefore, an image in which the glass tube 2 is expanded can be easily acquired by performing image compression photographing while rotating the glass tube 2 around the tube axis La.

The manufacturing process of the glass tube 2 includes a cutting process of cutting the glass tube 2 from the base glass 15 and an etching process of etching one end of the glass tube 2 after cutting. Since the end portion 17b of the glass tube 2 is etched when manufacturing the glass tube 2, the surface of the end portion 17b of the glass tube 2 can be finished. In the imaging step, the test image Dh is captured so that at least one of the etched surface formed on one end 17b of the tube glass 2 and the cut surface formed on the other end 17a is captured. is preferred. In this way, even if a defect occurs on the end face during cutting or etching, it is possible to classify and extract the defect by secondary determination. Note that if the glass tube 2 is transparent, it is also easy to capture the test image Dh so that both end surfaces are reflected at the same time.

Further, in the tube glass inspection method, the test image Dh of the tube glass 2 determined to be defective in the first inspection process and the second inspection process is acquired as learning data, and learning of the learning model M is performed based on the learning data. There may be a learning step to perform. In this case, it is possible to sequentially update the learning model M by learning the image data of the glass tube 2 determined to be defective in the inspection process, so that the learning model M can be optimized. Therefore, it is more advantageous to ensure high determination accuracy.

In addition, this embodiment can be changed and implemented as follows. This embodiment and the following modified examples can be implemented in combination with each other within a technically consistent range.
- In the secondary judgment, when the "reliability" is required as one parameter of the classification result, this reliability may be used for the judgment of the defect classification. For example, classification results with high reliability are validated and classification results with low reliability are invalidated, thereby improving the reliability of the classification itself. In this case, this further contributes to improving the accuracy of quality determination of the tube glass 2 .

As long as the rule-based image processing algorithm P is a rule-based program, various forms and schemes can be adopted.
Re-determination is not limited to processing using the rule-based image processing algorithm P, and may be changed to other processing such as processing using learning.

The re-determination process may be performed by the primary determination unit 7, for example.
• Rejudgment may include visual inspection by an inspector to confirm implementation.
- The learning model M is not limited to being generated by deep learning, and various machine learning methods can be applied.

- Classification types may include other types than the examples.
As long as the index F for image processing is a parameter that changes the image, various parameters such as image size and image saturation can be used.

- The updating of the learning model M may be omitted.
- The manufacturing method of the tube glass 2 may employ processes other than the cutting process and the etching process described in the embodiments. Moreover, the glass tube 2 may be produced through any manufacturing process.

- The glass tube 2 can be used as a glass material in various fields such as medical applications and lighting applications. Also, the tube glass 2 can be applied in various sizes, from fine ones to those with a certain size.

DESCRIPTION OF SYMBOLS 1... Tube glass inspection apparatus, 2... Tube glass, 3... 1st inspection part, 4... 2nd inspection part, 8... Backlight, 9... Camera, 10... Line sensor, 15... Base material glass, 17a, 17b... Dh... test image, Dk... learning image, P... image processing algorithm, M... learning model, F... index, R... threshold value, La... tube axis.

Claims (11)

A tube glass inspection method for inspecting the quality of tube glass,
an image capturing step of capturing an image to be inspected by developing the surface of the tubular glass;
a first inspection step of inspecting quality of the glass tube by identifying the image to be inspected by a rule-based image processing algorithm;
a second inspection step of reinspecting the quality of the glass tube by identifying the glass tube determined to be defective in the first inspection process using a learning model ;
In the second inspection process, the glass tube determined to be defective in the first inspection process is classified into the type of defect of the defective product through identification by the learning model, and the glass tube having the classified defect is classified into A glass tube inspection method for re-inspecting the quality of the glass tube by performing identification using a rule-based image processing algorithm in which a threshold value is set for each type of defect . 2. The tube glass inspection method according to claim 1 , wherein the types of defects are at least one of dents, rough skin, chips, scratches and stains. The learning model is generated by machine learning using a test image of the defective tube glass as a learning image,
the learning image is modified and duplicated by processing one learning image based on an image processing index;
3. The tube according to claim 2 , wherein a plurality of types of indices for the image processing are prepared, and an index corresponding to a classification of defects captured in one learning image to be processed is selected and used. Glass inspection method. 4. The tube glass inspection method according to claim 3 , wherein the index for image processing is any one of vertical inversion, horizontal inversion, rotation, brightness change, contrast change, angle change, position shift, or a combination thereof. If the defect captured in the single learning image to be processed is classified as a defect that occurs only at the end of the glass tube, left-right reversal and rotation are excluded from the image processing index. 5. The tube glass inspection method according to claim 4 , wherein the index is selected by wherein, in the imaging step, the tube glass is arranged between a backlight and a camera, and the image to be inspected is obtained by taking an image with the camera while rotating the tube glass around the tube axis. 6. The tube glass inspection method according to any one of 5 . The imaging region of the camera is a region longer than the tube glass in the axial direction of the tube glass.
The tube glass inspection method according to claim 6. a cutting step of cutting the tubular glass from the continuously molded base glass;
After cutting, an etching step of etching one end of the tube glass,
2. In the imaging step, the test image is captured so that at least one of an etched surface formed at one end of the tube glass and a cut surface formed at the other end of the tube glass is captured. 8. The tube glass inspection method according to any one of 7 . a learning step of acquiring, as learning data, an image to be inspected of the glass tube determined to be defective in the first inspection step and the second inspection step, and executing learning of the learning model based on the learning data; The tube glass inspection method according to any one of claims 1 to 8 . After an inspection for judging the quality of the tube glass by identifying the quality of the tube glass by identifying the image to be inspected by developing the surface of the tube glass with a rule-based image processing algorithm, the product is determined to be defective in the inspection. The glass tube is classified into defect types of defective products through identification by a learning model, and the glass tube classified with defects is identified by a rule-based image processing algorithm in which a threshold is set for each defect type. A learning method of the learning model used when re-inspecting the quality of the tube glass by performing
an image generation step of processing a single learning image based on image processing indices to create a plurality of test images for each of the indices from the single learning image;
a model generation step of generating the learning model from machine learning using the plurality of learning images as image data;
In the image generation step, in generating a plurality of learning images from a single learning image, a learning method in which an index corresponding to the classification of defective products is selected and used from among the plurality of indices. . A tube glass inspection device for inspecting the quality of tube glass,
a first inspection unit that acquires an inspection image of the surface of the glass tube developed and identifies the inspection image by a rule-based image processing algorithm to inspect the quality of the glass tube;
a second inspection unit that re-inspects the quality of the glass tube by identifying the glass tube determined to be defective by the first inspection unit using a learning model ;
The second inspection unit classifies the types of defects of the defective products through identification by the learning model with respect to the glass tube determined to be defective by the first inspection unit, and classifies the glass tube for which defects have been classified A glass tube inspection apparatus for re-inspecting the quality of the glass tube by performing identification using a rule-based image processing algorithm in which a threshold value is set for each type of defect .

Images