JP2023137057A - Method of generating defect prediction model, bottle appearance inspection method and bottle appearance inspection device - Google Patents

Abstract

To provide a method for generating a defect prediction model for performing highly accurate appearance inspection, a bottle appearance inspection method, and a bottle appearance inspection device.SOLUTION: A method for generating a defect prediction model includes: a learning image acquisition step of acquiring an image for learning which predicts the presence or absence of a defect appearing in the appearance of a bottle; a generation step of generating the teacher data by adding the defect information to the image for learning, and a learning step of generating a defect prediction model for predicting the presence or absence of the defect in an inspection target image when the inspection target image is inputted by performing the machine learning using the teacher data.SELECTED DRAWING: Figure 9

Description

The present invention relates to a method for generating a defect prediction model for inspecting the presence or absence of defects that appear on the exterior of glass bottles of various embodiments using images obtained by imaging the bottles, a method for inspecting the appearance of bottles, and a method for inspecting the appearance of bottles. Regarding equipment.

BACKGROUND OF THE INVENTION There are various types of defects in bottles, such as foreign objects, bubbles, and dirt, and it is known to inspect these bottle defects using an image of the bottle captured by a camera. Conventionally, in order to identify bottle defects on a gray-scale image, the positional relationship between a camera and an illumination device with respect to the bottle is set so that the camera can receive transmitted light from the bottle and obtain a transmitted image of the bottle. Defects appear as shadows in the transmitted image, and the area (number of pixels) of the low-brightness shadow area is measured, and bottle defects are detected based on the size of the area.

Among bottle defects, bubbles that appear near the surface of the bottle (hereinafter referred to as "surface bubbles") are difficult to detect from a transmitted image. Therefore, in order to identify surface bubbles on a gray-scale image, the positional relationship between the camera and the lighting device with respect to the bottle is set so that the camera can receive reflected light from the bottle and obtain a reflected image of the bottle. Surface bubbles appear in a transmitted image as an area that appears brighter than the surrounding area, rather than as a shadow.The area (number of pixels) of the area where the brightness is higher than a predetermined value is measured, and the surface bubbles are determined based on the size of the area. is being detected.

In recent years, advances in AI-related technology have been remarkable, and visual inspection devices have been proposed that use artificial intelligence to identify scratches and defects on objects. Patent Document 1 discloses a defect inspection device that uses an image signal obtained by scanning an inspection target to determine the shape and grade of a defect using artificial intelligence. Furthermore, in Patent Document 2, a trained model is created by deep learning using image data of the object as learning data, and using the trained model, the image data obtained by imaging the inspection object is used to create a trained model in advance. An image identification system is disclosed that identifies image data representing an image in a set format.

Japanese Patent Application Publication No. 5-280960 Japanese Patent Application Publication No. 2021-157735

By the way, there are various types of bottles, such as bottles with uneven deformed parts on the surface, such as ramune bottles, and bottles with reliefs such as company names, product names, decorations, etc. on the surface. be. For this type of bottle, when inspecting the quality of the bottle by identifying defects that appear on the bottle's appearance on an image, deformed parts and relief parts appear as shadows on the image, so the shadows caused by defects are It is necessary to distinguish it from shadows caused by relief or relief.

However, it is difficult to conduct a high-precision visual inspection of these types of bottles, which is not affected by deformed or relief parts, and the problem is that non-defective products are incorrectly judged as defective, resulting in an increase in the number of non-defective products that are discarded. . If the inspection accuracy is lowered in order to reduce the number of discarded non-defective products, defective products will be erroneously determined as non-defective products, resulting in the problem of defective products being put on the market.

Hitherto, no high-precision bottle appearance inspection apparatus that is not affected by deformed parts or relief parts, and furthermore, a bottle appearance inspection apparatus that uses artificial intelligence to determine defects in bottles that appear on the outside, have not been proposed.

The present invention has been made in view of the above-mentioned problems, and aims to provide a method for generating a defect prediction model, a bottle appearance inspection method, and a bottle appearance inspection device for performing highly accurate appearance inspection. .

A method for generating a defect prediction model for predicting the presence or absence of defects appearing on the appearance of a bottle according to the present invention includes a learning image acquisition step of acquiring a learning image;
a generation step of adding defect information to the learning image to generate teacher data;
and a learning step of generating a defect prediction model that predicts the presence or absence of a defect in the image to be inspected when the image to be inspected is input by performing machine learning using the teacher data.

According to the method of generating the defect prediction model described above, since defect information is added to the learning image, highly accurate visual inspection can be performed.

Preferably, the defect information includes information indicating a position of a defect within the learning image.

The defect information includes information indicating the type of defect, and the defect type is preferably at least one of scratches, surface bubbles, wrinkles, foreign matter, dirt, inner bubbles, and foreign glass.

The bottle visual inspection method of the present invention includes an acquisition step of acquiring an inspection target image of a bottle to be inspected, and a defect prediction model generated by the defect prediction model generation method described above. a defect prediction step of predicting the presence or absence of a defect.

In the above-mentioned bottle visual inspection method, in the defect prediction step, the defect prediction model may output, as a prediction result, a degree of reliability indicating the possibility of whether or not a defect is included in the image to be inspected. preferable.

The above-mentioned bottle visual inspection method preferably further includes a first determination step of determining whether or not there is a defect in the image to be inspected by comparing the reliability with a threshold value.

Regarding the inspection target image determined to have a defect in the first determination step, it is further determined whether or not the inspection target image has a defect by processing based on the brightness difference between each pixel constituting the image and surrounding pixels. It is preferable to further include a second determination step.

Describing the present invention from an apparatus perspective, the bottle visual inspection apparatus includes an inspection target image acquisition unit that acquires an inspection target image of a bottle to be inspected, and a defect prediction model generated by the defect prediction model generation method described above. A defect prediction unit that uses a model to predict the presence or absence of a defect in the image to be inspected.

In the bottle visual inspection apparatus, it is preferable that the defect prediction model outputs a degree of reliability indicating the possibility of whether or not there is a defect in the image to be inspected as a prediction result.

It is preferable that the image forming apparatus further includes a first determination unit that determines the presence or absence of a defect in the image to be inspected by comparing the reliability with a threshold value.

a second determination unit that determines whether or not the inspection target image has a defect based on a brightness difference between a pixel of interest and its surrounding pixels in the inspection target image determined to have a defect by the first determination unit; It is preferable to further include.

The bottle visual inspection apparatus of the present invention further includes an imaging device for capturing the inspection target image.

The imaging device includes a table that supports the bottle in an upright position and rotates the bottle around a central axis, an illumination device that irradiates the bottle with illumination light, and a table that receives the illumination light that is irradiated toward the bottle. The camera includes a camera that captures the inspection target image of the bottle on the table, and a control unit that controls at least an imaging operation of the camera, and the control unit includes a shutter that is capable of capturing an image over at least the entire circumference of the bottle. It is preferable to generate a timing signal that provides the timing of and output it to the camera.

According to the above configuration, highly accurate visual inspection can be performed.

1 is a plan view showing a schematic configuration of a bottle inspection system including a bottle appearance inspection device of the present invention. (A) is a front view showing an example of a bottle having a deformed portion, and (B) is a front view showing an example of a bottle having a relief portion. An example of a schematic configuration of a bottle visual inspection device is shown, with (A) being a plan view and (B) being a front view. Another example of the schematic configuration of the bottle visual inspection device is shown, in which (A) is a plan view and (B) is a front view. This is an example of a captured image of the bottle in FIG. 2(A). FIG. 2 is a block diagram showing the configuration of a defect prediction model generation device. FIG. 2 is a block diagram showing the configuration of a defect inspection device. (A) to (D) are examples of captured images that are determined to be defective by the first determination section. 3 is a flowchart showing the flow of defect prediction model generation. It is a flowchart which shows the flow of defect determination. It is a figure which shows the result of the verification test of the determination performance of a defect prediction model.

Embodiments of the present invention will be described with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments described below.

(Schematic configuration of bottle inspection system 1)
FIG. 1 shows a schematic configuration of a bottle inspection system 1 including a bottle visual inspection apparatus 20 of the present invention. The illustrated bottle inspection system 1 has a configuration in which a plurality of inspection stations are arranged in a circle around a star wheel 2. Note that FIG. 1 shows the final inspection station TA and one other inspection station TB, and the illustration of the other inspection stations is omitted. A plurality of recesses 2a into which bottles 10 to be inspected are introduced are provided on the outer peripheral surface of the star wheel 2 at equal angles.

Bottles 10 conveyed from a bottle making machine (not shown) are supplied to the recess 2a of the star wheel 2 by the bottle supply mechanism 3. The star wheel 2 is intermittently fed at a predetermined angle in the direction indicated by the arrow in the figure by a rotational drive mechanism (not shown), whereby the bottles 10 for inspection are sequentially guided to each inspection station. At each inspection station, the inspection bottle 10 is inspected for predetermined items. The bottle visual inspection apparatus 20 of the present invention is placed at any of the above-mentioned inspection stations (for example, inspection station TB). After the bottles 10 are inspected for various items, the bottles 10 to be inspected are discriminated into good and defective by the discrimination mechanism 4 according to the inspection results. The bottles 10 to be inspected that the discrimination mechanism 4 discriminates as "good" are sent onto the carry-out conveyor and sent to the next process. Bottles 10 to be inspected that are discriminated as "defective" by the discrimination mechanism 4 are rejected by a reject mechanism (not shown) and collected on a table for disposal.

(bottle 10)
2(A) and 2(B) show an example of a bottle 10 to be inspected by the bottle visual inspection apparatus 20. The bottle 10 in FIG. 2(A) has a body 13, a bottom 15, and a mouth 16, and the upper part of the body 13 is constricted and includes a deformed portion 11 having an uneven surface. In the present embodiment, the deformed portion 11 and its vicinity are set as the inspection target area to inspect for defects, but the inspection target area may be other areas of the trunk 13. The bottle appearance inspection device 20 can also inspect a bottle 10 whose surface is embossed with a company name, product name, decoration, etc., as shown in FIG. 2(B). The relief portion 12 is represented by a protrusion that slopes upward from the surface of the body 13 of the bottle 10, that is, a protrusion that has a chevron-shaped cross section. Note that the bottle 10 to be inspected is not limited to the example shown in FIG. 2, and a bottle 10 that does not have the deformed portion 11 or the relief portion 12 may be inspected.

Defects that appear on the bottle 10 include scratches on the surface of the bottle 10, bubbles that appear near the surface of the bottle 10 (also called "surface bubbles"), wrinkles, and undissolved objects such as metal or ceramic pieces ("foreign objects"). ), dirt adhering to the surface of the bottle 10 during the bottle making process, bubbles appearing inside the inner wall of the bottle 10 (bubbles other than surface bubbles, also referred to as "inner bubbles"), glass bottle 10 It is glass that has different components from those used as raw materials for glass, and is contained in recycled glass collected from the market, and is contaminated during the manufacturing of glass bottles using recycled glass ("foreign glass"). ) is included. When surface foam and internal foam are not distinguished, they are also referred to as "foam." In this specification, defects in the bottle 10 refer to scratches, surface bubbles, wrinkles, foreign matter, dirt, inner bubbles, and foreign glass that cause problems when using the bottle 10, and defects that do not cause problems when using the bottle 10. are not included in defects.

The bottle 10 shown in FIGS. 2(A) and 2(B) is an example of a glass bottle 10 to be inspected, but it may also be an example of a glass bottle 10 used as a learning object. The glass bottle 10 to be inspected is referred to as a "test bottle 10", and the glass bottle 10 used as a learning target is referred to as a "learning bottle 10". The learning bottle 10 and the testing bottle 10 may be collectively referred to simply as "bottle 10."

(Bottle appearance inspection device 20)
As shown in FIGS. 3 and 4, the bottle visual inspection device 20 of the present invention includes an imaging device 30 for capturing an image of the bottle 10, and the presence or absence of defects in the inspection target image captured by the imaging device 30. It also includes a defect inspection device 50 for inspecting.

(Imaging device 30)
As shown in FIG. 3, the imaging device 30 includes a table 31 that supports the bottle 10 in an upright position and rotates the bottle 10 around a central axis X, an illumination device 32 that irradiates the bottle 10 with illumination light, and The camera 33 captures an image of the bottle 10 on the table 31 by receiving illumination light directed toward the bottle 10 , the imaging operation of the camera 33 , the lighting and extinguishing operations of the illumination device 32 , the operation of the table 31 , etc. It also includes a control section 34 for controlling.

The imaging device 30 captures an image of the test bottle 10 (also referred to as an "inspection target image") when the test bottle 10 is inspected by the bottle visual inspection device 20. Furthermore, when an inspection is not being performed, it is also possible to capture an image of the learning bottle 10 (also referred to as a "learning image") for use in a defect prediction model generation device 40, which will be described later. When the learning image or the inspection target image is not distinguished, it is also simply referred to as a "captured image."

The table 31 is provided horizontally at the position of the inspection station TB, and can be rotated around a central axis X (axis) by a drive mechanism 31a. The table 31 and the star wheel 2 constitute a positioning mechanism for the bottle 10. The illumination device 32 irradiates the imaging target region 14 of the body 13 of the bottle 10 positioned at the inspection position with diffused light.

The camera 33 is disposed at a position facing the bottle 10 and at a position where it can receive transmitted light from the bottle 10 and obtain a captured image of the imaging target area 14 of the bottle 10. The imaging target area 14 is an area of a predetermined height of the body 13 of the bottle 10, and is a region of the entire circumference of the bottle 10 (the rotation angle of the bottle 10 is 360 degrees) and more than two circumferences (the rotation angle of the bottle 10 is 720 degrees). degree), more preferably an area that spans more than the entire circumference and less than a half of the entire circumference (the rotation angle of the bottle 10 is 540 degrees), and is an area that is acquired as an image captured by the camera 33. At the start and end of the rotation of the bottle 10, the bottle 10 on the table 31 (described later) shakes, etc., making the bottle 10 unstable, and defects may not be clearly imaged. In order to reliably obtain an image of the entire circumference of the bottle 10, it is preferable to image an area that spans at least the entire circumference of the bottle 10, and more preferably an area that spans about half the circumference of the bottle 10.

As the camera 33, in one example, a CCD line sensor in which a plurality of pixels are arranged in a vertical line is used. Note that the camera 33 is not limited to a line sensor. For example, an area camera may be used as the camera 33, the image capturing target area 14 may be imaged multiple times, and the captured images may be stitched together to form a captured image including the image capturing target area 14.

The camera 33 is controlled such that the shutter remains open during at least one revolution, preferably one and a half revolutions of the bottle 10, in order to obtain image data over at least the entire circumference, preferably the entire circumference and a half of the bottle 10. Ru. The control unit 34 generates a timing signal that provides timing for opening and closing the shutter, and outputs it to the camera 33.

FIG. 5 shows an expanded image taken by the camera 33 of the bottle 10 shown in FIG. The expanded captured image in FIG. 5 shows an expanded imaging area over the entire circumference (540 degrees) of the bottle 10, and images C1 to C3 of foreign objects as defects appear in three locations. In reality, only one foreign object is contained in the bottle 10. Images C1 to C3 of the foreign object appearing in the captured image of FIG. 5 are images of the same foreign object in the bottle 10 that was imaged three times. In the captured image of FIG. 5, the left end is the imaging start position and the right end is the imaging end position, and images C1 and C3 of the foreign object appearing at the left end and right end are the same rotation angle position of the foreign object after the bottle 10 rotates once. This is what appeared when the image was taken when I arrived. The foreign object image C2 is an image of the same foreign object captured from the back side of the bottle 10 when the bottle 10 rotates half a revolution from the position where the leftmost foreign object image C1 was acquired.

When inspecting defects such as foreign matter, dirt, inner bubbles, or foreign glass, the camera 33 is located at a position facing the bottle 10 and can receive transmitted light from the bottle 10 to obtain an image of the bottle 10. placed in position. In this embodiment, as shown in FIGS. 3(A) and 3(B), the camera 33 and the illumination device 32 are arranged on a straight line at opposing positions with the bottle 10 in between. Images of foreign objects, dirt, internal bubbles, and foreign glass defects are more likely to appear in captured images when transmitted light is used. In particular, the illumination device 32 includes a diffused light source and a polarizing plate, and by irradiating the bottle 10 with linearly polarized illumination light, images of foreign glass tend to appear in the captured image. Note that by using transmitted light, images of defects such as relatively large surface bubbles and relatively large scratches may also appear in the captured image.

When inspecting surface bubbles, scratches, and wrinkles as defects, the camera 33 is placed at a position facing the bottle 10 and at a position where it can receive reflected light from the bottle 10 and obtain an image of the bottle 10. be done. In this embodiment, as shown in FIG. 4(A), the camera 33 has a central angle of 45 mm with respect to the illumination device 32 in a plan view about a point 10a that includes the imaging location on the surface of the body 13 of the bottle 10. It is placed in the position of degree. Note that the central angle of the arrangement position of the camera 33 is not limited to this embodiment; for example, the central angle may be greater than or equal to 30 degrees and less than or equal to 60 degrees, and the camera 33 may The center angle may be any center angle as long as it can obtain enough diffused light to capture an image. Images of defects such as surface bubbles, scratches, and wrinkles are more likely to appear in captured images when reflected light is used. In addition, by using reflected light, defects can be detected such as foreign objects that reflect illumination light (e.g. metal exposed on the bottle surface), dirt that reflects illumination light (e.g. white dirt), and defects on bottles that reflect illumination light. 10 Internal bubble defects near the surface may also appear in the captured image. In addition, in FIG. 4(A) and FIG. 4(B), only the bottle 10, the table 31, the lighting device 32, and the camera 33 are shown, and illustration of other components is omitted.

A bottle 10 that is determined to have a defect in the imaging target area 14 by the inspection by the bottle appearance inspection device 20 is determined to be a "defective bottle 10" or a "defective product", and a bottle 10 that is determined to have no defect in the imaging target area 14 is determined to be a "defective bottle 10" or a "defective product". Bottle 10 is also referred to as "good bottle 10" or "good product."

The first bottle visual inspection device 20, in which a camera 33 and a lighting device 32 are arranged as shown in FIG. By providing second bottle visual inspection devices 20 arranged as shown in FIG. 4, the bottle 10 is imaged using both transmitted light and reflected light, and is inspected for scratches, surface bubbles, wrinkles, foreign matter, dirt, inner bubbles, and It is possible to inspect foreign glass for defects. Furthermore, if one inspection station has a sufficiently large space, two lighting devices 32 can be arranged for one camera 33 in the bottle visual inspection device 20 as shown in FIGS. 3 and 4. It's okay. In this case, the bottle 10 is rotated one or more times to capture an image using one of the transmitted light and the reflected light, and the bottle 10 is further rotated one or more times to capture an image using the other of the transmitted light and the reflected light.

In this embodiment, the control unit 34 is composed of a general-purpose computer, and may also serve as a general-purpose computer configuring the defect inspection apparatus 50 as shown in FIG. Alternatively, a dedicated computer may be used.

(Defect prediction model generation device 40)
Before explaining the specific functions of the defect inspection device 50, the defect prediction model generation device 40 used in the defect inspection device 50 will be explained. FIG. 6 is a block diagram showing a schematic configuration of the defect prediction model generation device 40. As shown in FIG. The defect prediction model generation device 40 is for generating a defect prediction model M for predicting the presence or absence of defects appearing on the appearance of the inspection bottle 10. The defect prediction model generation device 40 can be configured with a general-purpose computer, and its hardware configuration includes a processor such as a CPU or a GPU, a main storage device such as a DRAM or SRAM (not shown), and an HDD or SSD. A storage section 44 is provided. The storage unit 44 stores various programs, parameters, etc. for operating the defect prediction model generation device 40, such as a teacher data generation program and a learning program.

The learning image is obtained by capturing an image of the learning bottle. The learning image is an image for generating teacher data, and is image data of the imaging target region 14 of the learning bottle 10. An image of the defective bottle 10 is used as the learning image. Note that the learning image may include only the image of the non-defective bottle 10, or may include both the image of the non-defective bottle 10 and the image of the defective bottle 10. The number of learning images is not particularly limited, but is preferably a number that allows machine learning to be performed sufficiently. The learning bottle may be imaged in advance by a turntable, a camera, or the like provided separately from the imaging device 30, and the learning image may be stored in the storage unit 44, which will be described later. Further, as described above, when the bottle visual inspection device 20 is not performing an inspection, the imaging device 30 may be used to capture an image.

The defect prediction model generation device 40 includes the components shown as functional blocks in FIG. 6, and includes a learning image acquisition section 41, a defect information adding section 42, and a learning section 43. In this embodiment, each of these units is realized in software by the processor of the defect prediction model generation device 40 reading out the teacher data generation program and the learning program into the main storage and executing them.

The learning image acquisition unit 41 acquires learning images. In this embodiment, a learning image of the learning bottle 10 is acquired from the imaging device 30. Note that if the learning image is stored in the storage section 44 in advance, the learning image acquisition section 41 reads out the learning image from the storage section 44 and acquires it. Further, the learning image acquisition unit 41 may acquire learning images via an external communication line or a storage medium.

The defect information adding unit 42 adds defect information to generate teacher data. Defect information is information regarding defects. In the learning image of a defective product, the defect information includes information indicating the position of the defect within the learning image. The user visually specifies the defective area in the learning image by operating the input device 60, such as a mouse, touch panel, or monitor. For example, in FIG. 5, the user specifies an area surrounded by dotted lines. The defect information adding unit 42 adds the coordinate data of the designated area to the learning image as information indicating the position of the defect within the learning image. In addition, for small scratches, surface bubbles, wrinkles, foreign objects, dirt, inner bubbles, and foreign glass that appear in the training images and are not defects, the user should not specify the area as a defect or provide information indicating the position. Information indicating that there is no defect is given to the learning unit 43, which will be described later, to learn not to recognize small foreign objects, bubbles, thin dirt, etc. that are not defects as defects.

Note that the information indicating the position of the defect in the learning image may be provided by, for example, coloring pixels indicating the defect in the learning image, and inputting coordinate data where the defect is located into the input device 60. It may be given by Further, the defect information may include information indicating the type of defect. The types of defects are at least one of scratches, surface bubbles, wrinkles, foreign matter, dirt, inner bubbles, and foreign glass. Furthermore, the defect information may be information indicating the presence or absence of a defect in the learning image.

In addition, instead of being visually provided with the defect information through a user's operation, the defect information may be provided without the user's intervention. For example, the storage unit 44 may include a program for detecting defects on an image and a program for determining the type of defect, and the defect information may be provided by operating these programs.

The learning image to which defect information has been added by the defect information adding unit 42 is stored in the storage unit 44 as teacher data.

The learning unit 43 performs machine learning using teacher data to generate a defect prediction model M that predicts the presence or absence of a defect in the image to be inspected when the image to be inspected is input. Furthermore, the prediction result output by the defect prediction model M may be a reliability level that indicates the possibility of whether a defect exists or not, or may be information that only indicates whether a defect exists or not. , position information of the defect in the image to be inspected, and information regarding the type of defect (scratches, surface bubbles, wrinkles, foreign matter, dirt, internal bubbles, and foreign glass). Although the learning method is not particularly limited, for example, deep learning, support vector machine, random forest, etc. can be used. The generated defect prediction model M is stored in the storage unit 44 and then used for visual inspection of the bottle 10.

Note that when the learning image includes only images of non-defective bottles 10, the defect information adding unit 42 may add defect information indicating that there is no defect to the learning image.

In this embodiment, the defect prediction model generation device 40 may be provided on the cloud or may be configured by a general-purpose computer. Further, the defect inspection device 50 described later may have the function of the defect prediction model generation device 40.

(Defect inspection device 50)
FIG. 7 is a block diagram showing a schematic configuration of the defect inspection device 50. The defect inspection device 50 uses the defect prediction model M generated by the defect prediction model M generation method to determine whether or not there is a defect in the inspection target image of the bottle 10 captured by the imaging device 30. The defect inspection device 50 can be configured with a general-purpose computer, and has a hardware configuration including a processor such as a CPU or GPU, a main storage device (not shown) such as a DRAM or SRAM, and a storage unit such as an HDD or SSD. It is equipped with 56.

In addition to the defect prediction model M generated by the defect prediction model generation device 40 and the defect prediction model generation method, the storage unit 56 stores various programs for operating the defect inspection device 50 such as a defect prediction program and a defect determination program. and parameters are stored. Further, after the image to be inspected is captured by the imaging device 30, it may be stored in the storage unit 56.

The defect inspection apparatus 50 includes each component represented as a functional block in FIG. , and an output section 55. In this embodiment, each of these units is realized in software by the processor of the defect inspection device 50 reading various programs into the main storage device and executing them.

The inspection target image acquisition unit 51 acquires an inspection target image. In this embodiment, an inspection target image obtained by capturing an image of the inspection bottle 10 from the imaging device 30 is acquired. Note that the inspection target image may be captured in advance and stored in the storage unit 56, and in this case, the inspection target image acquisition unit 51 reads the inspection target image from the storage unit 56 and acquires it. Further, the inspection target image acquisition unit 51 may acquire the inspection target image via an external communication line or a storage medium.

The defect prediction unit 52 is a functional block that uses the defect prediction model M to predict the presence or absence of a defect in the image to be inspected. In this embodiment, the defect prediction unit 52 reads the defect prediction model M from the storage unit 56 into the main storage device. The defect prediction unit 52 inputs the image to be inspected to the defect prediction model M, and the defect prediction model M predicts the reliability indicating the possibility of whether or not there is a defect in the image to be inspected. output as the result. The defect prediction unit 52 inputs the reliability data output by the defect prediction model M to the first determination unit 53 as a prediction result. Note that the defect prediction model M may output reliability for each type of defect. That is, reliability levels are output that indicate whether or not the image to be inspected contains foreign matter, surface bubbles, bubbles other than surface bubbles, and dirt. Further, the defect prediction model M may output information indicating the position of the defect on the image to be inspected. Further, the defect prediction model M may output only the information of "presence" or "absence" of defects in the image to be inspected, instead of the reliability. In that case, the defect prediction unit 52 may output the information to the first determination unit 53 or the output unit 55.

The first determination unit 53 determines whether or not there is a defect in the image to be inspected by comparing the reliability output from the defect prediction unit 52 with a threshold value. The first determination unit 53 reads the threshold value from the storage unit 56, and determines that there is a defect in the image to be inspected when the reliability is higher than the threshold value, and determines that there is no defect in the image to be inspected when it is less than or equal to the threshold value. The first determination unit 53 outputs as a determination result whether or not there is a defect in the image to be inspected. Note that when the defect prediction model M outputs reliability for each type of defect, the first determination unit 53 makes a determination for each type of defect, and determines whether foreign matter, surface bubbles, bubbles other than surface bubbles, or dirt. Outputs the determination result as to whether there is a defect. If there is a defect, the determination result is output to the second determination section 54, and if there is no defect, the determination result is output to the output section 55.

FIGS. 8A to 8D are examples of images to be inspected that are determined to have defects by the first determination unit 53. Note that FIGS. 8A to 8D do not show the entire image to be inspected, but are images obtained by cutting out only a portion of the image to be inspected in which a portion determined to be a defect appears. In the figure, the dark or white parts within the area surrounded by dotted lines are parts determined to be defects. The dotted line is added for explanation and is not included in the actual image to be inspected. Surface bubbles appear in the region indicated by the dotted line in FIG. 8(A), and foreign matter appears in the region indicated by the dotted line in FIG. 8(B). On the other hand, light dirt on the surface of the bottle 10 appears in the dotted line area in FIG. 8(C), and small foreign matter appears in the dotted line area in FIG. 8(D). Small foreign objects, bubbles, thin stains, etc. as shown in FIGS. 8(C) and 8(D) may be determined as defects by the defect prediction unit 52 and the first determination unit 53, but depending on the quality required. There is no need to judge it as a defect.

Note that when only the information of "presence" or "absence" of a defect in the image to be inspected is inputted to the first judgment section 53 as a prediction result, the judgment result of the first judgment section 53 is "presence" or "absence" of a defect. It will be in line with the information "No."

The second determination unit 54 performs mask scanning on the inspection target image determined to have a defect by the first determination unit 53, and performs gradation processing based on the brightness difference between the pixel of interest and surrounding pixels in the mask to create a defect in the inspection target image. Determine whether or not there is.

The image to be inspected is a grayscale image in which the brightness of each pixel is represented by 256 gradations, and the darker the image, the smaller the brightness value. The second determination unit 54 performs processing (gradation processing) for each pixel to calculate the brightness difference with other pixels in the vicinity separated by a predetermined number of pixels. Subsequently, the second determination unit 54 reads the grayscale threshold value from the storage unit 56 to the main storage device, detects pixels whose brightness difference value exceeds the grayscale threshold value in the grayscale image, and converts the binary image into a binary image by binarization processing. generate. In this binary image, pixels exceeding the grayscale threshold are set as "1", and other pixels are set as "0". The second determination unit 54 counts the number of consecutive pixels among the pixels that exceed the gray threshold. For example, for a somewhat large foreign object, the pixels that make up the outline of the image of the foreign object are "1", the pixels inside and outside the outline are "0", and the pixels that make up the outline are continuous in a ring, and the outline The number of pixels forming the contour (corresponding to the circumference of the foreign object) is counted by tracking. Note that instead of the outline of the foreign object, the pixels surrounded by the outline may be counted as pixels constituting the foreign object (corresponding to the area of the foreign object). The second determination unit 54 reads the perimeter threshold (size threshold) from the storage unit 56 and compares the counted number of pixels with the size threshold. If the counted number of pixels is equal to or greater than the size threshold, it is determined that the image to be inspected has a defect. Even if the small foreign matter, bubbles, thin dirt, etc. shown in FIGS. 8(C) and 8(D) are determined to be defects by the first determining unit 53, the second determining unit 54 determines that they are not defects. It is determined that

Note that the second determination unit 54 does not scan the entire image to be inspected that has been determined to have a defect by the first determination unit 53 to determine the presence or absence of a defect, but only the area including the defect and the area around the defect. The presence or absence of a defect may be determined by scanning. In this case, the defect prediction unit 52 causes the defect prediction model M to output position information of the defect in the image to be inspected, and the second determination unit 54 acquires the position information from the first determination unit 53 or from the defect prediction unit 52. Thereby, the amount of calculation by the second determination section 54 can be reduced. In addition, when the defect prediction model M outputs the type of defect in the image to be inspected as a prediction result, the storage unit 56 stores in advance a grayscale threshold value and a size threshold value according to the type of defect, and the second determination unit 54 The presence or absence of a defect may be determined using a shade threshold and a size threshold depending on the type of defect.

The output unit 55 outputs the determination result of the presence or absence of a defect by the second determination unit 54 or the determination result of the absence of a defect by the first determination unit 53 to the discrimination mechanism 4 . Further, the output unit 55 may output the type of defect in the image to be inspected.

In this embodiment, the defect inspection device 50 may be provided on the cloud.

(Method for generating defect prediction model M)
The processing procedure of the method for generating the defect prediction model M according to this embodiment will be explained using the flowchart shown in FIG. The method for generating the defect prediction model M is executed by the defect prediction model generation device 40.

In the learning image acquisition step of step S1, the learning image acquisition unit 41 acquires a learning image from the imaging device 30. Subsequently, in the generation step of step S2, the defect information adding section 42 adds defect information to the learning image by the user's operation of the input device 60, and generates teacher data.

One piece of teacher data is generated from one learning image by steps S1 and S2. Steps S1 and S2 are repeated until all predetermined learning images have been processed, and when all the learning images have been processed (YES in step S3), the process moves to the learning step of step S4. In step S4, the learning unit 43 generates a defect prediction model M by performing machine learning based on the teacher data.

(Appearance inspection method for bottle 10)
The processing procedure of the visual inspection method for the bottle 10 according to this embodiment will be explained using the flowchart shown in FIG. The visual inspection method for the bottle 10 is performed by the imaging device 30 and the defect inspection device 50 of the bottle visual inspection device 20.

In step S11, the test bottle 10 is introduced into a predetermined test station TB of the bottle visual inspection device 20, and it is determined whether the test bottle 10 is placed on the table 31 of the imaging device 30. When the test bottle 10 is introduced (YES in step S11), the control unit 34 of the imaging device 30 turns on the illumination device 32 in step S12, rotates the table 31 in step S13, and turns on the camera 33 in step S14. Execute imaging. When the imaging is completed, the control unit 34 turns off the illumination and stops the rotation of the table 31 in step S15.

In the inspection target image acquisition step of step S16, the inspection target image acquisition unit 51 of the defect inspection apparatus 50 acquires the inspection target image. In the defect prediction step of step S17, the defect prediction unit 52 reads the defect prediction model M from the storage unit 56 to the main storage device, uses the defect prediction model M to predict the presence or absence of defects in the image to be inspected, and Output the degree as the prediction result. Step S18 is a first determination step, and the first determination unit 53 reads the threshold value from the storage unit 56, and determines that there is a defect in the image to be inspected when the reliability is higher than the threshold, and when the reliability is less than or equal to the threshold. It is determined that there is no defect in the image to be inspected, and the determination result is output. If it is determined that there is a defect in the image to be inspected, the process moves to step S19. If it is determined that there is no defect in the image to be inspected, the process moves to step S22. In step S22, the output unit 55 outputs a determination result that the bottle 10 associated with the image to be inspected is a good product to the discrimination mechanism 4.

In step S19, the second determination unit 54 performs predetermined image processing on the inspection target image determined to have a defect, and in the second determination step of step S20, determines whether the inspection target image has a defect. Determine whether or not. If it is determined in the second determination step that there is a defect, the output unit 55 outputs a determination result that the inspection bottle 10 associated with the image to be inspected is defective to the discrimination mechanism 4 in step S21. If it is determined that there is no defect, the process moves from step S20 to step S22, and the output unit 55 outputs to the discrimination mechanism 4 the determination result that the inspection bottle 10 associated with the inspection target image is a non-defective product. The step of visual inspection of one test bottle 10 is completed through steps S11 to S22, and the process returns to step S11 to inspect the next test bottle 10. When the inspected test bottle 10 is sent out from the table 31 and a new test bottle 10 is introduced onto the table 31, the appearance inspection in steps S11 to S22 is performed. If the inspection bottle 10 is not introduced in S11 and the inspection is finished (YES in S25), the operation of visual inspection of the bottle 10 is finished.

According to this embodiment, the defect prediction model M of the bottle 10 is generated using a learning image in which defects appear on the appearance of the bottle 10, so that highly accurate appearance inspection is possible. In particular, by generating the defect prediction model M using images of the bottle 10 having the deformed part 11 and the bottle 10 having the relief part 12 as learning images, This enables highly accurate visual inspection that is not affected by 12.

Further, in this embodiment, the first determination unit 53 uses the defect prediction model M to predict the presence or absence of a defect in the inspection target image, and for the inspection target image determined to have a defect, a second The determination unit 54 determines whether or not there is a defect in the image to be inspected based on the measurement of the brightness difference between each pixel constituting the grayscale image and surrounding pixels. Even if the defect prediction model erroneously determines the inspection target image related to a non-defective bottle in the first determination unit 53 as the inspection target image related to a defective bottle, the second determination unit 54 Since defect determination is performed again using a different determination method, it is possible to prevent an image to be inspected related to a good bottle from being mistakenly determined to be an image to be inspected related to a defective product, and a more highly accurate appearance inspection is possible.

(Verification test of judgment performance of defect prediction model M)
A verification test was conducted to determine whether the defect prediction model M can determine an image to be inspected as having a defect. First, a defect prediction model M was created using the defect prediction model generation device 40. Three hundred learning bottles 10 shown in FIG. 2(A) were prepared. There are 200 defective learning bottles 10 with defects. A learning image of each bottle 10 was captured by the imaging device 30, and defect information was added. A defect prediction model M was created using these 300 learning images.

The operation of the defect prediction model M described above was tested. As bottles 10 for inspection, 75 defective bottles 10 and 75 good bottles 10 were prepared. Each of these inspection bottles 10 was imaged by the imaging device 30 to obtain an image to be inspected, and the defect prediction model M was caused to output reliability. Figure 11 shows the test results. The horizontal axis of the graph in FIG. 11 is the bottle number assigned to each of the defective bottles 10 and non-defective bottles 10 to be inspected, and the vertical axis is the reliability. The closer the reliability is to 1, the higher the possibility that the image to be inspected has a defect.

Based on the reliability, it was determined whether the image to be inspected had a defect. The threshold value was set to 0.45, and bottles 10 associated with images to be inspected with a reliability of 0.45 or higher were defined as defective products, and bottles 10 associated with images to be inspected with a reliability of less than 0.45 were defined as non-defective products. The determination results are shown in Table 1. All 75 of the inspection target images (also referred to as "non-defective images") of the non-defective bottle 10 had a reliability lower than 0.45 and were determined to be non-defective. Furthermore, of the 75 images to be inspected (also referred to as "defective product images") of the defective product bottles 10, 83% of the images were determined to be defective products and 17% of the images were determined to be non-defective products. In this way, in determining the defects of the bottles 10 using the defect prediction model M, no non-defective product images were erroneously determined as defective products, and 83% of the defective product images were correctly determined as defective products. Note that 17% of defective products are determined to be non-defective products, but even if they are incorrectly determined to be non-defective products, a further visual inspection is performed after the inspection by the bottle appearance inspection device 20, so there is a risk that defective products will flow into the market. It can be suppressed reliably.

Further, Table 2 shows the determination results when the threshold value was set to 0.4. Of the 75 non-defective images, 99% of the images were determined to be non-defective, and only 1% were erroneously determined to be defective. Furthermore, regarding images of defective products, 85% were determined to be defective products and 15% were determined to be non-defective products. As described above, there were few images that were incorrectly determined to be defective even though they were non-defective images, and 85% of the images of defective products were correctly determined to be defective. An image that is erroneously determined to be a defective product even though it is a non-defective image is judged again by the second determination unit 54 and is determined to be a non-defective product.

Although one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various changes can be made without departing from the spirit of the present invention. The dimensions, materials, shapes, relative arrangements, etc. of the components described in the embodiments or shown in the drawings are not intended to limit the scope of the invention thereto, and are merely illustrative examples. The expressions "comprising," "comprising," "comprising," "containing," or "having" one component are not exclusive expressions that exclude the presence of other components.

1 Bottle inspection system 2 Star wheel 4 Discrimination mechanism 10 Bottle 11 Deformed portion 12 Relief portion 14 Imaging target area 20 Bottle visual inspection device 30 Imaging device 31 Table 32 Illumination device 33 Camera 34 Control unit 40 Defect prediction model generation device 41 Learning image Acquisition unit 42 Defect information provision unit 43 Learning unit 50 Defect inspection device 51 Inspection target image acquisition unit 52 Defect prediction unit 53 First determination unit 54 Second determination unit

Claims (13)

A method for generating a defect prediction model for predicting the presence or absence of defects appearing on the appearance of a bottle, the method comprising:
a learning image acquisition step of acquiring a learning image;
a generation step of adding defect information to the learning image to generate teacher data;
a learning step of generating a defect prediction model that predicts the presence or absence of a defect in the image to be inspected when the image to be inspected is input by performing machine learning using the training data; Generation method. 2. The defect prediction model generation method according to claim 1, wherein the defect information includes information indicating a position of a defect within the learning image. 3. The defect information includes information indicating the type of defect, and the type of defect is at least one of scratches, surface bubbles, wrinkles, foreign objects, dirt, inner bubbles, and foreign glass. How to generate a defect prediction model. an acquisition step of acquiring an image to be inspected of the bottle to be inspected;
A bottle comprising: a defect prediction step of predicting the presence or absence of a defect in the image to be inspected using the defect prediction model generated by the defect prediction model generation method according to any one of claims 1 to 3. Appearance inspection method. 5. The bottle visual inspection method according to claim 4, wherein in the defect prediction step, the defect prediction model outputs a degree of reliability indicating the possibility of whether or not there is a defect in the image to be inspected as a prediction result. 6. The bottle visual inspection method according to claim 5, further comprising a first determination step of determining whether or not there is a defect in the image to be inspected by comparing the reliability with a threshold value. Regarding the inspection target image determined to have a defect in the first determination step, it is further determined whether or not the inspection target image has a defect by processing based on the brightness difference between each pixel constituting the image and surrounding pixels. The bottle visual inspection method according to claim 6, further comprising a second determination step. an inspection target image acquisition unit that acquires an inspection target image of a bottle to be inspected;
A defect prediction unit that predicts the presence or absence of a defect in the image to be inspected using the defect prediction model generated by the defect prediction model generation method according to any one of claims 1 to 3;
Bottle visual inspection equipment including: 9. The bottle visual inspection apparatus according to claim 8, wherein the defect prediction model outputs a degree of reliability indicating the possibility of whether or not there is a defect in the image to be inspected as a prediction result. The bottle visual inspection device according to claim 9, further comprising a first determination unit that determines whether or not there is a defect in the image to be inspected by comparing the reliability with a threshold value. a second determination unit that determines whether or not the inspection target image has a defect based on a brightness difference between a pixel of interest and its surrounding pixels in the inspection target image determined to have a defect by the first determination unit; The bottle visual inspection device according to claim 10, further comprising: The bottle visual inspection device according to any one of claims 8 to 11, further comprising an imaging device for capturing the inspection target image. The imaging device includes a table that supports the bottle in an upright position and rotates the bottle around a central axis, an illumination device that irradiates the bottle with illumination light, and a table that receives the illumination light that is irradiated toward the bottle. including a camera that acquires the inspection target image of the bottle on the table, and a control unit that controls at least the imaging operation of the camera;
13. The bottle visual inspection apparatus according to claim 12, wherein the control unit generates a timing signal that provides timing for a shutter that can capture images over at least the entire circumference of the bottle, and outputs the generated timing signal to the camera.

Images