TWI797145B - Defect inspection system and defect inspection method - Google Patents

Abstract

In a defect inspection system 1, a two-dimensional image F (t1) or the likes whose luminance changes in a conveying direction X is imaged by an imaging unit 3, and processed by a line division processing unit 9 to an image data of a line division image DL1 (t1) which is formed of lines L1 (t1) or the likes at the same position in the two-dimensional image F (t1) or the likes arranged in a time serial order, the line division images DL1 (t1) having different luminance are classified into a plurality of line divided image groups G1 (t1) by a classifying unit 10, pixel values of the image data obtained by imaging the same position of an inspection target T at the line division image DL1 (t1) or the likes are integrated by an integration unit 11, an image integration data C1 (t1) or the likes is generated for each line division image group G1 (t1) or the likes, and the image integration data C1 (t1) is divided and outputted in accordance with a predetermined rule by a division output unit 12, whereby the appearance of the images of the image integration data C1 (t1) of the likes are different for each division according to the predetermined rule, and the type of the defect D is easily recognized.

Description

Defect inspection system and defect inspection method

The present invention relates to a defect inspection system and a defect inspection method.

As a defect inspection system that inspects defects of an inspection object based on captured images of the inspection object, for example, defect inspection for detecting defects in optical films such as polarizing films and retardation films, laminated films used in battery separators, and the like is known. system. Such a defect inspection system transports a film along a transport direction, captures two-dimensional images of the film at discrete times, and performs defect inspection based on the captured two-dimensional images. For example, the system of Japanese Patent No. 4726983 generates column-segmented images by dividing the two-dimensional image into a plurality of columns arranged along the conveying direction, and making the two-dimensional images captured at discrete times Columns at the same position in each of the images are arranged in time-series order. The column-segmented image is processed into a defect-enhanced processed image in which brightness variations are enhanced. For the defect enhancement processed image, the values of the pixels of the image material of the defect enhancement processed image obtained by photographing the same position of the inspection object are accumulated and integrated. Thereby, the presence or absence of a defect in the film and the position of the defect in the film can be easily identified.

In addition, there are types of film defects such as protruding convex defects and depressed concave defects. In the defect inspection system, it is desired to identify not only the presence or absence of defects and the position of defects, but also the type of defects. However, in the above technique, the values of the pixels of the image data of the defect enhancement processing image are accumulated and merged, so that the protrusion and depression information of the defect disappears or cancels each other out, and it is difficult to identify the type of the defect, and improvement is desired. .

Therefore, an object of the present invention is to provide a defect inspection system and a defect inspection method that can easily identify the type of defect.

The present invention relates to a defect inspection system, comprising: a light source for irradiating light to an inspection object; object or reflected light on the inspection object; the conveying unit transports the inspection object relative to the light source and the imaging unit along the conveying direction; The image data of the two-dimensional image is processed, and the imaging unit captures a two-dimensional image whose brightness changes in a direction consistent with the conveying direction of the two-dimensional image. The two-dimensional image is divided into a plurality of rows arranged along the conveying direction, and the two-dimensional image is processed into the image data of the row-divided image. Columns at the same position in each of the three-dimensional images are arranged in the order of time series; the classification unit classifies each column segmented image processed by the column segmentation processing unit into two or more columns according to preset rules a group of segmented images; the merging unit is configured to combine pixel values of image data obtained by photographing the same position of the inspection object for each column segmented image classified into the same column segmented image group by the classification unit, The image group is divided into columns to generate image combination data; and the segmentation output unit divides and outputs the image combination data generated by dividing the image group into columns according to preset rules.

According to this configuration, the defect inspection system includes: a light source for irradiating light onto the inspection object; It is formed by the light reflected on the inspection object; the conveying part is to convey the inspection object relative to the light source and the imaging part along the conveying direction; and the image processing part is to process the two-dimensional image taken by the imaging part The image data of the image is processed, wherein, the two-dimensional image whose brightness changes in the direction consistent with the conveying direction of the two-dimensional image is captured by the imaging unit, and the two-dimensional image is divided by the column division processing unit of the image processing unit. The two-dimensional image is divided into a plurality of columns arranged along the conveying direction, and the two-dimensional image is processed into the image data of the column-divided image. Columns at the same position in each of the two-dimensional images are arranged in time-series order. Therefore, even if the image is taken of the same inspection object, the divided images of each column have different brightness. picture. In addition, the classification unit of the image processing unit classifies the column segmented images with different luminances processed by the column segment processing unit into two or more column segmented image groups according to preset rules, and for the classification into For each column segmented image of the same column segmented image group, the combining unit of the image processing unit combines the pixel values of the image data obtained by photographing the same position of the inspection object to form the column segmented image group. The image combination data is generated, and the image combination data generated by dividing the image group by column by the combination unit is divided and output by the segmentation output part of the image processing part according to the preset rules, so even the pixels of the image data The image merged data obtained by merging the values of each other will also be segmented and output according to the preset rules, so that the presentation of the images of the image merged data will be different according to the segmentation based on the preset rules. Therefore, It is easy to recognize the type of defect of the inspection target in the image merged data.

In this case, it is preferable that the two-dimensional image captured by the imaging unit at discrete times has a bright part, a dark part, and a boundary between the bright part and the dark part, and the transport unit moves the object to be inspected relative to the imaging unit along the border between the bright part and the bright part. , the dark part and the boundary part intersect the conveying direction to be conveyed relatively.

According to this configuration, the two-dimensional image captured by the imaging unit at discrete times has a bright portion, a dark portion, and a boundary between the bright portion and the dark portion, and the transport unit moves the object to be inspected along with the bright portion, the dark portion, and the bright portion with respect to the imaging unit. The conveying directions intersecting the boundary portions are relatively conveyed, so that each part of the inspection object in a series of two-dimensional images captured at discrete times enters any one of the bright portion, the dark portion, and the boundary portion. Therefore, even if the image data output by the segmentation output unit is image integration data in which the values of the pixels of the image data are merged, the presentation method of the image of the image integration data will be divided based on a preset rule. And it is further different, so it is easier to identify the type of defect of the inspection object in the combined image data.

In this case, it is preferable that the classification unit classifies each column divided image processed by the column division processing unit into a column division obtained by arranging the columns of bright parts in each of the two-dimensional images in order of time series. a column-divided image group of images; a column-divided image group of column-divided images formed by arranging columns of dark portions in each of the two-dimensional images in a time-series order; The column-divided image group of the column-divided image formed by arranging the columns of the boundary part in the order of time series.

According to this configuration, the classification unit classifies each column segmented image processed by the column segmentation processing unit into a column segmented image in which the columns of the bright part in each of the two-dimensional images are arranged in time-series order. The column division image group of the image; the column division image group of the column division image formed by arranging the columns of the dark part in each of the two-dimensional images in the order of time series; and the column division image group of each of the two-dimensional images The column division image group of the column division image in which the columns of the boundary part are arranged in the order of time series, so even if the image data output by the division output part is an image obtained by combining the values of the pixels of the image data In the combined data, because the image group is divided and output according to the columns of the bright part, the dark part, and the boundary part, the image presentation method of the image combined data is further different by dividing the image group by column, so it is more It is easy to recognize the type of defect of the inspection target in the image merged data.

In addition, the defect inspection system may further include a light-shielding body that is positioned between the light source and the inspection object and that blocks part of the light irradiated from the light source to the inspection object, so that the two images captured by the imaging unit at discrete times A bright portion, a dark portion, and a boundary portion are formed on the three-dimensional image, and the conveying unit relatively transports the inspection object relative to the light source, the light-shielding body, and the imaging portion along a conveying direction intersecting the bright portion, the dark portion, and the boundary portion.

According to this structure, the bright part, the dark part, and the boundary part can be easily formed on the two-dimensional image by the light-shielding body, and each part of the inspection object in a series of two-dimensional images captured at discrete times can enter the bright part, the dark part, and the boundary part. Any of the dark part and the boundary part.

In addition, it is preferable that the segmentation output unit divides the image combination data generated by dividing the image group by column by the combination unit into pixels as follows, and outputs each pixel as an image obtained by photographing the same position of the inspection object. The pixels of the merged data and the brightness of the bright part, the dark part and the boundary part are above an arbitrary bright threshold; the pixels of the image merged data obtained by shooting the same position of the inspection object and are the bright part and the dark part and the pixel whose brightness at the boundary is below any dark threshold; and it is the pixel of the image combined data obtained by shooting the same position of the inspection object and is the magnitude of the brightness change at the bright part, the dark part and the boundary is any pixel above the change threshold.

According to this configuration, the image integration data generated by dividing the image group in columns by the integration unit is divided into pixels by the segmentation output unit into pixels and outputted: Pixels that are bright, dark, and boundary parts are always bright pixels whose brightness is above an arbitrary bright threshold; they are pixels of image merged data obtained by shooting the same position of the inspection object and are bright and dark and a pixel whose brightness at the boundary is always dark below an arbitrary dark threshold; and it is a pixel of the combined image data obtained by photographing the same position of the inspection object and is the brightness at the bright part, the dark part, and the boundary The magnitude of the change is above any change threshold, which is rich in bright and dark pixels, so it is easy to divide the image group by column to grasp the presentation of the bright part, dark part and boundary part of the image of the image merged data, so it is easier It is easy to recognize the type of defect of the inspection target in the image merged data.

In this case, it is preferable that the division output unit divides the image integration data generated by dividing the image group by column by the integration unit into a coloring unit that colors each pixel in a color different from each other, and outputs each pixel. The pixels of the image combined data obtained by photographing the same position of the inspection object are the pixels whose brightness at the bright part, dark part, and boundary part is above an arbitrary bright threshold; it is the image obtained by photographing the same position of the inspection object Pixels of image merged data and pixels whose brightness at the bright part, dark part, and boundary part are below an arbitrary dark threshold; Pixels in which the magnitude of change in luminance at the dark portion and the boundary portion is equal to or greater than an arbitrary change threshold.

According to this structure, the image integration data generated by dividing the image group by the column by the integration unit is divided into the coloring unit for coloring each pixel in a color different from each other by the division output unit, and outputs the output unit. The pixels of the image combination data obtained by shooting the same position of the inspection object are always bright pixels whose brightness at the bright part, dark part and boundary part is above an arbitrary bright threshold; The pixels of the merged image obtained by shooting are always dim pixels whose brightness at the bright part, dark part and boundary part is below an arbitrary dark threshold; The pixel of the data is a pixel rich in bright and dark changes in which the magnitude of the brightness change at the bright part, dark part, and boundary part is greater than or equal to an arbitrary change threshold. Thus, by dividing the image group by column, the appearance of the bright part, dark part, and boundary part of the image in the combined image data can be easily grasped by color, so it is easier to identify the part of the inspection object in the combined image data. The category of the defect.

In addition, it is preferable that the division output unit divides and outputs the image integration data generated by dividing the image group by column by the merging unit by division of the image group by column.

According to this structure, even if the image data is the image data in which the values of the pixels of the image data are merged with each other, the representation of the image in the image data data is changed by dividing the image group by column and outputting it. Since the row division image group differs, it is easy to recognize the type of defect of the inspection target in the image merged data.

In addition, it is preferable that the segmentation output unit divides the image integration data generated by dividing the image group by column by the combining unit at a preset threshold value between the group of column divided images, and outputs it.

According to this structure, even if the image data is the image data in which the pixel values of the image data are merged with each other, the image data will be distorted due to the fact that the image group is segmented by the predetermined threshold value and output. The presentation of the images differs by a predetermined threshold value between the column-segmented image groups, so it is easy to identify the type of defect of the inspection object in the image merged data.

In addition, it is preferable that, for each column segmented image classified into the same column segmented image group, the merging unit determines the brightness of pixels of the image data obtained by capturing the same position of the inspection object relative to the reference value. A difference value with a sign is assigned to a pixel, and the difference values of the pixels of image data obtained by photographing the same position of the inspection object are merged together, and the image group is divided into columns to generate image merged data.

According to this configuration, for each column segmented image classified into the same column segmented image group, the merging unit performs an adjustment based on the brightness of the pixels of the image material obtained by capturing the same position of the inspection object relative to the reference value. Pixels are given difference values with positive and negative signs, and the difference values of pixels of image data obtained by photographing the same position of the inspection object are combined with each other, and the image group is divided into columns to generate image combination data, so it is possible to use By simple calculation, the group of images is divided into columns to generate combined image data in which the luminance of pixels of the image data captured at the same position of the inspection object is enhanced relative to the reference value.

In addition, it is preferable that the merging unit enhances the luminance change of adjacent pixels of the image data obtained by capturing the same position of the inspection object for each column segmented image classified into the same column segmented image group. processing, and combine the enhanced values of the pixels of the image data obtained by photographing the same position of the inspection object, and divide the image group by column to generate the combined image data.

According to this configuration, the merging unit executes an enhancement process of enhancing the luminance change of adjacent pixels in the image material obtained by capturing the same position of the inspection object for each column segmented image classified into the same column segmented image group. , and combine the enhanced values of the pixels of the image data obtained by photographing the same position of the inspection object, and divide the image group by column to generate the combined image data. Therefore, it is possible to perform column-by-column The divided image group generates image combined data in which brightness changes of adjacent pixels of image data obtained by capturing the same position of the inspection object are enhanced.

In addition, it is preferable that the defect inspection system further includes an analysis unit for identifying the defects to be inspected based on data accumulated as a result of machine learning related to the recognition of the types of defects included in the image integration data. category, wherein the image integration data is the image integration data generated by dividing the image group by column by the integration unit.

According to this configuration, the analysis unit recognizes the inspection object based on the data obtained by accumulating the results of machine learning related to the recognition of the type of defect contained in the image combination data generated by dividing the image group by column by the combination unit. The type of defect, the image merged data obtained by combining the pixel values of the image data with each other divides the image group by column and merges the pixel values of the image data with each other, so based on the machine learning of the image merged data As a result, the type of defect can be identified, and the accuracy of identifying the type of defect to be inspected can be improved.

In another aspect, the present invention relates to a defect inspection method including: an irradiation step of irradiating light from a light source of a defect inspection system to an inspection object; and an imaging step of capturing two-dimensional images at discrete times by an imaging unit of the defect inspection system, wherein The two-dimensional image is based on the light that is irradiated from the light source to the inspection object in the irradiation process and penetrates the inspection object or is reflected on the inspection object; The conveying process in which the parts are relatively transported along the conveying direction; and the image processing process in which the image data of the two-dimensional image captured in the imaging process is processed by the image processing unit of the defect inspection system, in the imaging process , taking a two-dimensional image whose brightness changes in a direction consistent with the conveying direction of the two-dimensional image, the image processing process includes: dividing the two-dimensional image into a plurality of columns arranged along the conveying direction, And the column division processing step of processing the image data of the two-dimensional image into column-divided images, wherein the column-divided images are obtained by making the two-dimensional images captured at discrete times by the imaging unit the same The columns of positions are arranged in the order of time series; the classification process of classifying each column segmented image processed in the column segmentation process into two or more column segmented image groups according to preset rules; For each column segmented image classified into the same column segmented image group in the process, the pixel values of the image data obtained by capturing the same position of the inspection object are merged, and the image group is segmented by row to generate image merged data a merging process; and a segmentation output process of segmenting and outputting the image merging data generated by dividing the image group by column in the merging process according to a preset rule.

In this case, it is preferable that in the irradiation process, the two-dimensional image captured at discrete times in the imaging process has a bright part, a dark part, and a boundary part between the bright part and the dark part, and that the inspection object is opposed to each other in the transport process. The imaging unit is relatively transported along the transport direction intersecting the bright part, the dark part and the boundary part.

In this case, in the classifying step, it is preferable to classify the column-divided images processed in the column-dividing processing step so that the columns of bright parts in each of the two-dimensional images are arranged in time-series order. The column-segmented image group of the resulting column-segmented image; the column-segmented image group of the column-segmented image formed by arranging the columns of dark parts in each of the two-dimensional images in the order of time series; and the two-dimensional image A column-divided image group of column-divided images in which the columns of the boundaries in each image are arranged in time-series order.

In addition, it is preferable that in the irradiation process, a light-shielding body, which is located between the light source and the inspection object, is formed on the two-dimensional image captured at discrete times in the imaging process by a light-shielding body, and the light-shielding body is positioned between the light source and the inspection object, and the Part of the light irradiated from the light source to the inspection object is blocked, and in the conveying process, the inspection object is relatively conveyed with respect to the light source, light shielding body, and imaging unit along the conveying direction intersecting the bright portion, the dark portion, and the boundary portion.

In addition, it is preferable that in the segmentation output process, the image integration data generated by dividing the image group by column in the integration process is divided into the following pixels and output as follows: each pixel is the same position of the inspection object. The pixels of the image combination data obtained by shooting are the pixels whose brightness at the bright part, the dark part and the boundary part is above any bright threshold value; it is the pixel of the image combination data obtained by shooting the same position of the inspection object and is Pixels whose luminance at the bright part, dark part and boundary part are below an arbitrary dark threshold; and which are the pixels of the combined image data obtained by photographing the same position of the inspection object and are the luminance at the bright part, dark part and boundary part The magnitude of the change is any pixel above the change threshold.

In this case, it is preferable that in the division and output process, the image integration data generated by dividing the image group by column in the integration process is divided into coloring parts in which each pixel is colored in a different color and output, and each pixel Refers to: it is the pixel of the image combination data obtained by shooting the same position of the inspection object, and the brightness of the bright part, dark part and boundary part is above an arbitrary bright threshold; it is the pixel of the same position of the inspection object The pixels of the merged image data obtained by photographing and the brightness of the bright part, dark part and boundary part are below an arbitrary dark threshold; and the pixels of the merged image data obtained by photographing the same position of the inspection object And it is a pixel whose brightness change width at the bright portion, the dark portion, and the boundary portion is equal to or greater than an arbitrary change threshold.

In addition, it is preferable that in the division output step, the image integration data generated by dividing the image group by column in the integration step is divided into the division image group by column and output.

In addition, it is preferable that in the segment output step, the image integration data generated by segmenting the image group by row in the combining step is segmented by a predetermined threshold between the row-by-row segment image groups and output.

In addition, it is preferable that in the merging step, for each column segmented image classified into the same column segmented image group, the difference between the luminance of the pixel of the image data obtained by capturing the same position of the inspection object with respect to the reference value High and low, to assign a difference value with a positive or negative sign to the pixel, combine the difference values of the pixels of the image data obtained by shooting the same position of the inspection object, and divide the image group by column to generate the combined image data .

In addition, it is preferable that in the merging step, for each column segmented image classified into the same column segmented image group, the luminance of adjacent pixels of image data obtained by capturing the same position of the inspection object is enhanced. Enhancement processing of changes, combining enhanced values of pixels of image data captured at the same position of the inspection object, and dividing the image group by column to generate image merged data.

In addition, it is preferable that the defect inspection method further includes an analysis step in which the inspection object is identified based on data accumulated as a result of machine learning related to the identification of the type of defect included in the combined image data. The category of the defect, wherein the image combination data is the image combination data generated by dividing the image group by row through the combination process.

1: Defect inspection system

2: light source

3: Camera Department

4: Conveying department

5: Image processing department

6: Shading body

7: parallel light lens

8: Analysis department

9: Column division processing unit

10: Classification Department

11: Merger Department

12: Split output unit

100: Convolutional Neural Networks

110: Input layer

120: hidden layer

121, 123: Convolution layer

122:Pooling layer

124: Fully connected layer

130: output layer

b: Boundary

C1, C2, C3: image merge data

D: defect

DL1, DLj, DLk: column segmentation image

d: Anbu

F: 2D image

G1, G2, G3: column-segmented image groups

L1, Lj, Lk: columns

l: bright department

R, B, G: Coloring department

S1: Irradiation process

S2: Camera process

S3: Conveying process

S4: Image processing process

S5: Analysis process

S41: Column division processing step

S42: Classification process

S43: Merging process

S44: Split output process

T: check object

t, t1, t2, t3, tm: time

X: conveying direction

Y: width direction

Fig. 1 is a perspective view showing a defect inspection system according to an embodiment.

FIG. 2 is a diagram showing the arrangement of a light source, an imaging unit, a light shield, and an inspection object in the defect inspection system of FIG. 1 .

Fig. 3 is a block diagram showing details of an image processing unit of the defect inspection system in Fig. 1 .

Fig. 4 is a flow chart showing the steps of the defect inspection method of the embodiment.

FIG. 5 is a flow chart showing details of the image processing process in FIG. 4 .

(A), (B), (C), (D), (E), (F), and (G) in Fig. 6 show the column division processing performed by the image processing unit of the defect inspection system in Fig. 1 A plot of the partially processed image.

(A) of Fig. 7 is a diagram showing a time-series two-dimensional image, (B) is a diagram showing a divided image of each column in which the columns of each position are arranged in the order of time series, and (C) is A diagram showing an alignment image obtained by shifting time so that each row of divided images in (B) shows the same position of the inspection object.

(A), (B) and (C) in Fig. 8 are diagrams showing images processed by the classification unit of the image processing unit in the defect inspection system in Fig. 1, and (D), (E) and (F ) is a diagram showing an image processed by the integration unit of the image processing unit of the defect inspection system in FIG. 1 .

Fig. 9 is a diagram showing a state where each pixel of the image integration data in (D) in Fig. 8 is output as a coloring part colored in different colors by the division output part of the defect inspection system in Fig. 1 .

Fig. 10 is a diagram showing a convolutional neural network.

Hereinafter, preferred embodiments of the defect inspection system and defect inspection method of the present invention will be described in detail with reference to the drawings. As shown in FIGS. 1 and 2, a defect inspection system 1 according to an embodiment of the present invention includes a light source 2, an imaging unit 3, a transport unit 4, an image processing unit 5, a light shielding body 6, a parallel light lens 7, and an analysis unit. 8. The defect inspection system of this embodiment uses optical films such as polarizing films and retardation films, laminated films used for battery separators, and the like as inspection objects T, and detects defects in the inspection objects T. The inspection object T extends along the conveyance direction X of the conveyance unit 4 , and has a predetermined width in a width direction Y perpendicular to the conveyance direction X. Defects generated in the inspection object T refer to a state different from the expected state, such as foreign matter, scratches, bubbles (bubbles generated during molding, etc.), foreign matter bubbles (bubbles generated by the mixing of foreign matter, etc.) ), scratches, cracks (cracks due to creases, etc.), and streaks (streaks due to differences in thickness, etc.). The defect inspection system 1 recognizes the categories of these defects.

As shown in FIGS. 1 and 2 , the light source 2 irradiates the inspection object T with light. The light source 2 is arranged so as to emit linear light parallel to the width direction Y. The light source 2 is not particularly limited as long as it is a metal halide lamp, a halogen transmission lamp, a fluorescent lamp, or the like that emits light that does not affect the composition and properties of the film to be inspected T.

The imaging unit 3 captures two-dimensional images at discrete times based on light irradiated from the light source 2 to the inspection object T and passed through or reflected on the inspection object T. The imaging unit 3 has a plurality of optical members and photoelectric conversion elements. The optical member includes an optical lens, a shutter, and the like, and forms an image of light transmitted through a film as the inspection target T on the surface of the photoelectric conversion element. The photoelectric conversion element is an area sensor composed of an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor) that captures a two-dimensional image. The imaging unit 3 may be a member that captures either a two-dimensional image without color or a two-dimensional image with color.

The transport unit 4 transports the inspection object T relative to the light source 2 and the imaging unit 3 along the transport direction X. The conveyance part 4 is equipped with the sending-out roller and the receiving roller which convey the film which is inspection object T along conveyance direction X, for example, and measures conveyance distance with a rotary encoder etc. In the present embodiment, the transport speed at which the transport unit 4 transports the inspection object T is set to about 2 to 100 m/min along the transport direction X. The conveying speed of the conveying unit 4 is set and controlled by the image processing unit 5 and the analyzing unit 8 .

The image processing unit 5 processes the image data of the two-dimensional image captured by the imaging unit 3 . The image processing unit 5 is not particularly limited as long as it is a member that performs image processing of two-dimensional image data, and for example, a PC (personal computer) equipped with image processing software, or an FPGA equipped with an image processing circuit can be applied. (Field Programmable Gate Array, Field Programmable Gate Array) image acquisition card, etc.

The light-shielding body 6 is located between the light source 2 and the inspection object T, and blocks a part of the light irradiated from the light source 2 to the inspection object T to form a bright portion on the two-dimensional image captured by the imaging unit 3 at discrete times. , the dark part, and the boundary between the light part and the dark part. Through the light-shielding body 6 , the imaging unit 3 captures a two-dimensional image whose brightness changes in a direction that coincides with the conveyance direction X of the two-dimensional image. More specifically, the transport unit 4 transports the inspection object T relative to the light source 2 , the collimator lens 7 , the light shield 6 and the imaging unit 3 along the transport direction X intersecting the bright portion, the dark portion, and the boundary portion. In this embodiment, the longitudinal direction of the boundary portion is parallel to the width direction Y perpendicular to the conveyance direction X. It should be noted that the imaging unit 3 does not need to include the light shielding body 6 as long as it can capture a two-dimensional image whose brightness changes in a direction that coincides with the conveyance direction X of the two-dimensional image. The collimator lens 7 parallelizes the traveling direction of light irradiated from the light source 2 to the inspection object T and the light shielding body 6 . The parallel light lens 7 can be constituted by a telecentric optical system, for example.

Hereinafter, details of the image processing unit 5 will be described. As shown in FIG. 3 , the image processing unit 5 includes a column division processing unit 9 , a classification unit 10 , an integration unit 11 , and a division output unit 12 . The row division processing section 9 processes the two-dimensional image into image data of a row-divided image. The row-divided image is obtained by dividing the two-dimensional image into a plurality of rows arranged along the conveying direction X, and making The columns at the same position in each of the two-dimensional images captured by the imaging unit 3 at discrete times are arranged in time-series order.

The classification unit 10 classifies each column divided image processed by the column division processing unit 9 into two or more column divided image groups according to a preset rule. The merging unit 11 merges pixel values of image data obtained by photographing the same position of the test object T for each column segmented image classified into the same column segmented image group, and generates a map for each column segmented image group. like merging data.

More specifically, the merging unit 11 classifies each column segmented image processed by the column segment processing unit 9 into a column segmented image in which the columns of bright parts in each of the two-dimensional images are arranged in time-series order. The column division image group of the image; the column division image group of the column division image formed by arranging the columns of the dark part in each of the two-dimensional images in the order of time series; and the column division image group of each of the two-dimensional images A column-divided image group of column-divided images in which the columns of the boundaries are arranged in time-series order.

In addition, for each column segmented image classified into the same column segmented image group, the merging unit 11 determines whether the pixel brightness of the image material obtained by capturing the same position of the test object T is higher or lower relative to the reference value. A difference value with a sign is assigned to a pixel, and the difference values of pixels of image data obtained by capturing the same position of the inspection object are combined to divide the image group into columns to generate the combined image data. Alternatively, the merging unit 11 performs an enhancement process of enhancing the luminance changes of adjacent pixels of the image data obtained by capturing the same position of the inspection object for each column segmented image classified into the same column segmented image group, In addition, values obtained by performing enhancement processing on pixels of image data captured at the same position of the inspection object are combined, and the image group is divided into rows to generate image merged data.

The segmentation output unit 12 divides and outputs the image combination data generated by dividing the image group by column by the combination unit 11 according to a preset rule. More specifically, the segmentation output unit 12 divides and outputs the image integration data generated by dividing the image group by column by the integration unit 11 by dividing the image group by column. Alternatively, the segmentation output unit divides and outputs the image integration data generated by dividing the image group by column by the merging unit by a preset threshold value between the group of column-segmented images. The preset threshold value is, for example, the brightness of a pixel of a column divided image belonging to a column divided image group, or the like.

In addition, the segmentation output unit 12 divides the image combination data generated by the column division image group by the combination unit 11 into the column division image group, divides it into the following pixels, and outputs the following pixels: The pixels of the combined image obtained by taking pictures at the same position of T and the pixels whose brightness at the bright part, dark part and boundary part are above any bright threshold value; The pixels of the data and the brightness at the bright part, dark part and boundary part are pixels below an arbitrary dark threshold; and the pixels of the image combined data obtained by shooting the same position of the inspection object T and are the bright part and dark part and the pixels whose luminance change width at the boundary portion is equal to or greater than an arbitrary change threshold.

Furthermore, the segmentation output unit 12 divides and outputs the image integration data generated by dividing the image group by column by the integration unit 11 into coloring portions in which each pixel is colored in a different color from each other, and outputs each pixel: The pixels of the merged image obtained by photographing at the same position of T are the pixels whose brightness at the bright part, dark part and boundary part is above any bright threshold; it is the merged image obtained by photographing the same position of the inspection object The pixels of the data and the brightness at the bright part, dark part and boundary part are pixels below an arbitrary dark threshold; and the pixels of the combined image obtained by shooting the same position of the inspection object and are the pixels of the bright part, dark part and Pixels in which the magnitude of change in luminance at the boundary portion is equal to or greater than an arbitrary change threshold.

Returning to FIG. 2 , the analysis unit 8 connected to the image processing unit 5 is constituted by, for example, a PC (personal computer). The analysis unit 8 recognizes the defects of the inspection target T based on the data obtained by accumulating the results of machine learning related to the recognition of the types of defects included in the image combination data generated by dividing the image group by column by the combination unit 11 . category. Data accumulating the results of machine learning are stored in a storage device such as a hard disk of a PC including the analyzing unit 8, and are updated according to the results of machine learning.

It should be noted that, in this embodiment, the data obtained by accumulating the results of machine learning related to the recognition of the types of defects included in the row segmented images include, in addition to the data related to the internal imaging unit of the defect inspection system 1 3 In addition to the data obtained by accumulating the results of machine learning related to the recognition of the types of defects contained in the series of segmented images after processing a series of two-dimensional images captured at discrete times, it also includes data related to defect inspection The data obtained by accumulating the results of machine learning related to the identification of the type of defects included in the column segmented images separately created outside the system 1 . That is, in this embodiment, in addition to the scheme of identifying the type of defect in the state where machine learning is performed in the defect inspection system 1, it also includes a method based on the state where machine learning has not been performed in the defect inspection system 1. It is a plan to identify the type of defect by accumulating the data obtained by accumulating the result of machine learning separately created outside the defect inspection system 1 .

In addition, the analysis unit 8 displays the type of the defect recognized by the image processing unit 5 on an LC (Liquid Crystal) display panel, a plasma display panel, an EL (Electro Luminescence) display panel, or the like. In addition, the image processing part 5 may recognize the type of a defect, and may have the analysis part 8 which displays the processed image inside.

Hereinafter, the defect inspection method of this embodiment will be described. As shown in FIG. 4 , an irradiation step ( S1 ) of irradiating light from the light source 2 of the defect inspection system 1 to the inspection target T is performed. As shown in (A) of FIG. 6 , in the irradiation process, the light-shielding body of the defect inspection system 1 that is located between the light source 2 and the inspection object T and blocks part of the light irradiated from the light source 2 to the inspection object T is used. 6. Form a bright portion l, a dark portion d, and a boundary portion b between the bright portion l and the dark portion d on the two-dimensional image F(t1) captured at discrete times in the imaging process. As shown in (A) of Fig. 6, for the two-dimensional image F(t1) at time t=t1, the light from the light source 2 is blocked by the light shielding body 6, so as it reaches the downstream side of the transport direction X On the other hand, the brightness in the two-dimensional image F(t1) becomes high. In addition, the defect D on the film of the inspection target T is projected on the two-dimensional image F(t1). The same applies to the two-dimensional images F(t2), F(t3), ..., F(tm) at time t=t2, t3, ..., tm (m is an arbitrary natural number).

As shown in FIG. 4, the imaging process (S2) is performed by the imaging unit 3 of the defect inspection system 1. In this imaging process, a two-dimensional image F(t1) is captured at a discrete time. The two-dimensional image F(t1) ) is formed based on the light that is irradiated from the light source 2 to the inspection object T and penetrates the inspection object T or is reflected on the inspection object T in the irradiation process. As shown in (A) of FIG. 6 , in the imaging process, a part of the light irradiated from the light source 2 to the inspection object T is blocked by the light shielding body 6 , so that the two-dimensional image F(t1) in the transport direction Two-dimensional image F(t1) whose brightness changes in the same direction as X. The same applies to the two-dimensional images F(t2), F(t3), . . . , F(tm) at times t=t2, t3...tm.

In addition, as shown in FIG. 4 , the conveying process ( S3 ) of relatively conveying the inspection object T relative to the light source 2 and the imaging unit 3 along the conveying direction X is performed by the conveying unit 4 of the defect inspection system 1 . As shown in (A) of FIG. 6 , in the transport process, the inspection object T is placed along the light source 2, the collimator lens 7, the light shielding body 6, and the imaging section 3 along the bright portion l, the dark portion d, and the boundary portion b. The intersecting conveying directions X convey oppositely. In this embodiment, the boundary portion b is parallel to the width direction Y perpendicular to the conveyance direction X, but the angle formed by the boundary portion b and the conveyance direction X may be an angle other than 90°. In addition, the boundary part b is not necessarily a strict boundary part, and the boundary part refers to the part where the brightness of the two-dimensional image F(t1) including the bright part l is the highest and the part where the brightness of the two-dimensional image F including the dark part d is the smallest. middle part.

k)。列L1(t1)~列Lk(t1)的輸送方向X的寬度與在時刻t1、時刻t2、...、時刻tj、...、時刻tm的各時刻中的一幀間隔中將 檢查對象T沿著輸送方向X輸送的距離相同。對時刻t=t2、t3...tm中的二維圖像F(t2)、F(t3)、...、F(tm)也進行同樣的處理。 As shown in FIG. 4, image processing for processing image data of two-dimensional images F(t1) to F(tm) captured in the imaging process is performed by the image processing unit 5 of the defect inspection system 1. Process (S4). Hereinafter, details of the image processing step will be described. As shown in FIG. 5 , in the image processing step, the column division processing unit 9 of the image processing unit 5 of the defect inspection system 1 performs a column division processing step ( S41 ). As shown in (B) of FIG. 6 , in the column division processing step, the column division processing unit 9 divides the two-dimensional image F(t1) into a first column L1 ( t1)~jth column Lj(t1)~kth column Lk(t1) (j and k are any natural numbers, j

k). The width of the conveying direction X of the column L1(t1)~the column Lk(t1) and the inspection object in one frame interval at each time of time t1, time t2, ..., time tj, ..., time tm T conveys the same distance along the conveying direction X. The same process is performed on the two-dimensional images F(t2), F(t3), . . . , F(tm) at times t=t2, t3...tm.

The column segmentation processing section 9 processes the two-dimensional images F(t1)~F(tm) into image data of column segmentation images. The column segmentation images are two-dimensional images captured at discrete times during the imaging process. Columns L1 ( t1 ), L1 ( t2 ), L1 ( t3 ) and the like at the same position in each of F( t1 ) to F(tm) are arranged in the order of time series. An example of the segmented image in the first column will be described. As shown in (C) of FIG. 6 , the column division processing unit 9 makes the transfer of the two-dimensional images F(t1), F(t2), F(t3), ... captured at discrete times The first row L1 ( t1 ), L1 ( t2 ), L1 ( t3 ), . As shown in (D) of FIG. 6 , the column division processing unit 9 makes the first columns L1 ( t1 ) to L1 ( tm ) in each of the two-dimensional images F ( t1 ) to F ( tm ) follow the time-series They are arranged sequentially to generate the first column of divided images DL1 ( t1 ).

As shown in (E) of FIG. 6, (F) of FIG. 6, and (G) of FIG. Do the same for the first column L1(t1)~L1(tm),..., the jth column Lj(t1)~Lj(tm),..., the kth column Lk(t1)~Lk(tm) The processing generates the first-column divided images DL1(t1), ..., the j-th column divided images DLJ(t1), ..., the k-th column divided images DLk(t1). As shown in (E) of FIG. 6, the column division image DL1(t1) is the columns L1(t1)~L1( tk) Column segmentation images arranged in the order of time series.

As shown in (A) of FIG. 7 and (B) of FIG. 7, the column division images DL1(t1)~DLk(t1) are two-dimensional images F(t1)~F taken at discrete times (tm) The column division image in which the columns L1(t1)~Lk(t1) at the same position are arranged in the order of time series, therefore the column division image DL1(t1) of the range at the same time ~DLk(t1) represents different positions of the inspection object T, and the positions of the defects D in the column division images DL1(t1)~DLk(t1) are also shifted, respectively. Therefore, in the present embodiment, by creating a column division image in which the columns at the same position in the two-dimensional images captured at different time ranges are arranged in the order of time series, the Alignment is performed so that each row of divided images shows the same position of the inspection object T. FIG.

As shown in (A) of FIG. 7 , in the imaging process, two-dimensional images F(t1) to F(tm) are captured at discrete times. Since the inspection object T is conveyed along the conveyance direction X, the positions of the defects D in the two-dimensional images F(t1) to F(tm) are respectively shifted. As shown in (B) of FIG. 7 , the column divided images DL1 ( t1 ) to DLj ( t1 ) to DLk ( t1 ) are generated as described above. The column division images DL1(t1)~DLk(t1) in the range of the same time represent different positions of the inspection object T, so the positions of the defects D in the column division images DL1(t1)~DLk(t1) are also respectively offset.

With respect to the first row L1(t1)~L1(tm) from the downstream side of the conveying direction X, for example, the jth row Lj(t1)~Lj( tm) displays the upstream position of the inspection object T in the conveyance direction X by the distance the inspection object T is to be conveyed at a frame interval corresponding to (j-1). Therefore, as shown in (C) of FIG. 7, with respect to the column division image DL1(tm) of the first column L1(tm)~L1(t(m+(m-1))), for example, the j-th column For the column segmented image, the time t(m-(j-1))~time t(m+( The column division image DLj(t(m-(j-1))) of the range of m-j)) shows the same position of the inspection object T.

Similarly, for the column division image DL1(tm) of the first column L1(tm)~L1(t(m+(m-1))), for example, for the column division image of the k-th column, relative to The range from time t1 to time tm is a column division image in the range from time t(m-(k-1)) to time t(m+(m-k)) obtained by tracing back the time equivalent to the frame interval of (k-1) DLk(t(m-(k-1))) shows the same position of the inspection object T.

Alternatively, with respect to the column division image DL1(t1) of the first column L1(t1)~L1(t(1+(m-1))), for example, for the column division image of the jth column, time t The column division image DLj(t(1-(j-1))) in the range from (1-(j-1)) to time t(1+(m-j)) shows the same position of the inspection object T. In addition, with respect to the column divided image DL1(t1) of the first column L1(t1)~L1(t(1+(m-1))), for example, for the column divided image of the k-th column, time t The column division image DLk(t(1-(k-1))) ranging from (1-(k-1)) to time t(1+(m-k)) shows the same position of the inspection object T. By shifting the range of time as described above, alignment can be performed so that each row of divided images shows the same position of the inspection object T. FIG.

For example, the column division image DLj(t(1-(j-1))) shown in (F) in Fig. 6 is a two-dimensional image F(t(1-(j-1)))~F The sequence Lj(t(1-(j-1)))~Lj(t(m-(j-1))) of the position of the boundary part b in (t(m-(j-1))) follows time Sequences are arranged sequentially to segment images by columns. In addition, the column division image DL(k-2)(t(1-(k-3))) shown in (G) of FIG. 6 is a two-dimensional image F(t(1-(k-3) )))~F(t(m-(k-3))) in the column L(k-2)(t(1-(k-3)))~Lk(t(m- (k-3))) Column segmentation images arranged in the order of time series.

It should be noted that, in the case where the amount of positional shift is known or the size of the column segmented image is sufficiently large relative to the defect, the defect must fall into the column segmented image, so even if no alignment is performed, it can be Using Column Segmented Images Containing Defects for Machine Learning. Therefore, in such a case, alignment need not be performed.

As shown in FIG. 5 , in the image processing step, the classification step is performed by the classification unit 10 of the image processing unit 5 of the defect inspection system 1 ( S42 ). In the classification step, the classification unit 10 classifies each column divided image DL1 ( t1 ) and the like processed in the column division step into two or more column divided image groups according to a predetermined rule.

As shown in (A) of FIG. 8 , in the classification step, the classification unit 10 classifies each column divided image DL1(t1) obtained in the column division processing step into two-dimensional images F(t1) The column segmentation diagrams of columns L1(t1)..., L2(t0)..., L3(t(-1))..., etc. of the bright part l in each of them are arranged in the order of time series Columns like DL1(t1), DL2(t0), DL3(t(-1)), . . . divide the image group G1(t1).

As shown in (B) of FIG. 8 , in the classifying step, the classifying unit 10 classifies each column divided image DLj(t(1-(j-1))) and the like processed in the column dividing processing step into Let the columns Lj(t(1-(j-1)))..., L(j+1 ( Columns of t(1-(j-1))), DL(j+1)(t(1-j)), DL(j+2)(t(1-(j+1)))... The group of images G2(t(1-(j-1))) is segmented.

As shown in (C) of FIG. 8 , in the classification step, the classification unit 10 divides each column divided image DL(k-2)(t(1-(k-3))) processed in the column division processing step Equal classification is the column L(k-2)(t(1-(k-3))) of the dark part d in each of the two-dimensional image F(t(1-(k-3))).. ., L(k-1)(t(1-(k-2))..., Lk(t(1-(k-1)))... and other columns arranged in the order of time series Segmented images DL(k-2)(t(1-(k-3))), DL(k-1)(t(1-(k-2))), DLk((1-(k-1 ))) The columns of ... segment the image group G3(t(1-(k-3))).

As shown in FIG. 5 , in the image processing step, the integration process is performed by the integration unit 11 of the image processing unit 5 of the defect inspection system 1 ( S43 ). As shown in (D) of FIG. 8 , in the merging step, the merging unit 11 performs the merging for the column divided images DL1 ( t1 ), DL2 ( t0 ), and DL3 classified into the same column divided image group G1 ( t1 ). (t(-1)), ..., combine the pixel values of the image data obtained by photographing the same position of the inspection object T, and divide the image group G1 (t1) into columns to generate the combined image data C1 (t1).

As shown in (E) of FIG. 8 , in the merging step, the merging unit 11 classifies the column divided images DLj( t(1-(j-1))), DL(j+1)(t(1-j)), DL(j+2)(t(1-(j+1)))..., will The pixel values of the image data obtained by shooting the same position of the inspection object T are merged with each other, and the image group G2 (t(1-(j-1))) is divided into columns to generate the image merged data C2(t(1 -(j-1))).

As shown in (F) of FIG. 8 , in the merging step, the merging unit 11 classifies the column divided images DL ( k-2)(t(1-(k-3))), DL(k-1)(t(1-(k-2))), DLk((1-(k-1))).. ., combine the pixel values of the image data obtained by shooting the same position of the inspection object T, and divide the image group G3 by column (t(1-(k-3))) to generate the combined image data C3( t(1-(k-3))).

In the merging step, the merging unit 11 may be configured to classify the column divided images DL1(t1), DL2(t0), DL3(t(-1) into the same column divided image group G1(t1), etc. )... etc., according to the brightness of the pixels of the image data obtained by shooting the same position of the inspection object T relative to the reference value, the pixels are given a difference value with a sign of positive and negative, and the inspection object T The difference values of the pixels of the image data captured at the same position of T are merged, and the image group G1(t1) etc. are divided into columns to generate the image merged data C1(t1) and the like.

Alternatively, in the merging step, the merging unit 11 may be configured to classify the column divided images DL1(t1), DL2(t0), DL3(t(− 1))... etc., implement enhancement processing to enhance the brightness changes of adjacent pixels of the image data obtained by shooting the same position of the inspection object T, and take the image data obtained by shooting the same position of the inspection object T The enhanced values of the pixels of the image data are merged, and the image group G1 ( t1 ) etc. is divided into columns to generate image merged data C1 ( t1 ) and the like. It should be noted that the examples shown in (D) of FIG. 8, (E) of FIG. 8 and (F) of FIG. The shape of the defect D etc. varies.

As shown in FIG. 5 , in the image processing step, the split output unit 12 of the image processing unit 5 of the defect inspection system 1 performs a split output step ( S44 ). In the segmentation output step, the segmentation output unit 12 divides and outputs the integrated image data C1(t1) etc. generated by dividing the image group G1(t1) etc. into columns in the integration step according to a preset rule. For example, in the segmentation output step, the segmentation output unit 12 divides the image group G1 (t1) etc. into columns by dividing the image group G1 (t1) etc. into the image integration data C1 (t1) etc. generated by dividing the image group G1 (t1) etc. Split and output. First, in the segmentation output process, the segmentation output unit 12 divides the image group G1 (t1) etc. into column-by-column image integration data C1 (t1) etc. generated by dividing the image group G1 (t1) etc. to split.

As shown in FIG. 9, in the division and output process, the division and output unit 12 outputs the following pixels respectively as a coloring part R colored in red, a coloring part B colored in blue, and a coloring part G colored in green. Each pixel refers to a pixel such as the combined image data C1(t1) obtained by photographing the same position of the inspection object T, and the luminance at the bright part l, dark part d, and boundary part b is above an arbitrary bright threshold value It is the pixel of the combined image data C1(t1) obtained by photographing the same position of the inspection object T, and is the pixel whose brightness at the bright part l, dark part d and boundary part b is below an arbitrary dark threshold and it is the pixels of the image combination data C1 (t1) etc. obtained by shooting the same position of the inspection object T, and the magnitude of the brightness change at the bright part l, the dark part d and the boundary part b is an arbitrary change threshold pixels above.

For example, the segmentation output unit 12 may use the pixels of the combined image data C1(t1) obtained by photographing the same position of the inspection object T as the bright threshold value for the brightness of the bright part l, dark part d, and boundary part b. Each of the above pixels that are always bright is assigned to the colored part R, which is the pixel of the image integration data C1(t1) obtained by photographing the same position of the inspection object T, and is defined as the bright part l, the dark part d, and the boundary part Each pixel whose brightness at b is below the dark threshold and is always dark is assigned to the colored part B, which is the pixel of the combined image data C1(t1) obtained by photographing the same position of the inspection object T as the bright part 1. Each pixel rich in bright and dark changes whose magnitude of change in brightness at the dark part d and the boundary part b is above the change threshold is assigned as the colored part G.

暗閾值。 The distribution of the coloring part R, the coloring part B, and the coloring part G can also be set arbitrarily. In addition, the bright threshold, dark threshold and change threshold can be set arbitrarily, but usually the bright threshold

dark threshold.

For example, in the division and output step, the division and output unit 12 may divide the image group G1 into columns such as the image integration data C1 ( t1 ) and the like generated by dividing the image group G1 ( t1 ) into columns in the integration step. (t1) and so on with a preset threshold to segment and output.

As shown in FIG. 4 , the analysis step ( S5 ) is performed by the analysis unit 8 of the defect inspection system 1 . The category of the defect D of the inspection object T is identified by combining the data obtained by accumulating the results of machine learning related to the identification of the category of the defect included in the data C1 ( t1 ). Machine learning is performed, for example, by convolutional neural networks. It should be noted that a neural network other than a convolutional neural network or other methods may be used as long as the type of defect can be identified by machine learning.

As shown in FIG. 10 , the convolutional neural network 100 includes an input layer 110 , a hidden layer 120 and an output layer 130 . The image processing unit 5 of the defect inspection system 1 divides the image groups G1(t1), G2(t(1-(j-1))), G3(t(1-(k-1)) and G3(t(1-(k- 3) The integrated image data C1(t1), C2(t(1-(j-1))), C3(t(1-(k-3))) generated by etc. are input to the input layer 110 . The hidden layer 120 has convolutional layers 121 and 123 for performing image processing based on the weight filter f, and a pooling layer 122 for performing processing to vertically and horizontally reduce the two-dimensional arrays output from the convolutional layers 121 and 123 to leave effective values, And update the fully connected layer 124 of the weight coefficient n of each layer. In the output layer 130 , the recognition result of the category of the defect D by machine learning is output. In the convolutional neural network 100 , the error between the output recognition result and the positive solution value is backpropagated to the reverse direction R to learn the weight of each layer.

For example, a plurality of image integration data C1(t1) is input to the analysis unit 8 together with a positive solution for identifying the type of the defect D in advance, and the analysis unit 8 is made to learn, thereby sequentially recognizing the newly input image integration data C1( t1) and so on whether the category included is a specific defect D category, and sequentially output the recognition results. The recognition results output sequentially and the error of the positive solution are reversely propagated to the reverse direction R, and the weight coefficient n of each layer is sequentially updated and accumulated as data. In the state where the weights of each phase are sequentially updated, it is further sequentially identified whether the category contained in the newly input image combination data C1(t1) etc. is a specific defect category, and the identification results are sequentially output, based on the sequentially output identification The weight coefficient n of each layer is sequentially updated according to the error between the result and the correct solution and accumulated as data, and this is repeated, so that the error between the recognition result and the correct solution becomes smaller, and the recognition accuracy of the category of the defect D is improved.

This embodiment relates to a defect inspection system 1 that includes: a light source 2 that irradiates light to an inspection object T; an imaging unit 3 that captures a two-dimensional image F(t1) at discrete times, The two-dimensional image F(t1) is formed based on the light irradiated from the light source 2 to the inspection object T and penetrates the inspection object T or is reflected on the inspection object T; The imaging unit 3 is relatively conveyed along the conveying direction X; and the image processing unit 5 is used to process the image data of the two-dimensional image F (t1) captured by the imaging unit 3, wherein the image captured by the imaging unit 3 is The two-dimensional image F(t1) whose brightness changes in the direction corresponding to the conveying direction X of the two-dimensional image F(t1) is divided into two-dimensional image F(t1) by the column division processing part 9 of the image processing part 5. ) process the image data of column-divided image DL1(t1), etc., by dividing the two-dimensional image F(t1) into a plurality of columns L1 arranged along the conveying direction X (t1) and so on and arrange the column L1(t1) at the same position in each of the two-dimensional images F(t1) captured by the imaging unit 3 at discrete times in a time-series order. Therefore, even for The images captured by the same inspection object T, the divided image DL1 ( t1 ) and the like for each column are also images having different luminances.

In addition, the classification unit 10 of the image processing unit 5 classifies the column division images DL1(t1) and the like having different luminances processed by the column division processing unit 9 into two or more columns according to a predetermined rule. For the segmented image group G1(t1) etc., for each column segmented image DL1(t1) etc. classified into the same column segmented image group G1(t1) etc., the combination section 11 of the image processing section 5 checks The pixel values of the image data obtained by shooting at the same position of the object T are merged with each other, and the image group G1(t1) etc. are divided into columns to generate image merged data C1(t1) etc., which are output by the division of the image processing unit 5 The unit 12 divides and outputs the combined image data C1(t1) etc. generated by dividing the image group G1(t1) etc. by the column by the combining unit 11 according to preset rules, so that even the pixel values of the image data The image integration data C1(t1) etc. obtained by combining each other are divided and output according to a preset rule, thereby making the image integration data C1(t1) etc. The presentation of the images is also different, so that it is easy to recognize the type of the defect D of the inspection object T in the integrated image data C1 ( t1 ) and the like.

In addition, according to the present embodiment, the two-dimensional image F(t1) captured by the imaging unit 3 at discrete times has a bright part l, a dark part d, and a boundary part b between the bright part l and the dark part d, and the transport part 4 The inspection object is relatively transported relative to the imaging unit 3 along the transport direction X intersecting the bright part l, the dark part d and the boundary part b, so a series of two-dimensional images F(t1).. Each part of the inspection object T in . enters any one of the bright part l, the dark part d and the boundary part b. Therefore, even if the image data output by the segmentation output unit 12 is an image integration data obtained by combining pixel values of the image data, the image of the image integration data obtained by dividing based on a preset rule The presentation methods of will be further different, so it is easier to identify the category of the defect D of the inspection object T in the combined image data.

In addition, according to the present embodiment, each column segmented image processed by the column segment processing unit 9 is classified by the classifying unit 10 so that the column of the bright portion 1 in each of the two-dimensional image F(t1) and the like is time-dependent. The column division image group G1(t1) of the column division image DL1(t1) arranged in sequential order, and the columns of the dark part d in each of the two-dimensional image F(t1) are arranged in the order of time series The column division image group G3(t(1-(k-3))) of the column division image DL(k-2)(t(1-(k-3))), and the two-dimensional image Column division image group G2(t (1-(j-1))), so even if the image data output by the segmentation output unit 12 is an image merged data formed by combining the values of the pixels of the image data, it will also be divided by the bright part 1 and the dark part. The fact that the row segmented image group G1(t1) etc. of the boundary part b is segmented and output makes the presentation of the image of the image integration data further different for each column segmented image group G1(t1) etc. The type of the defect D of the inspection target T in the integrated image data can be easily recognized.

In addition, according to the present embodiment, a series of two-dimensional images captured at discrete times can be easily formed by the light-shielding body 6 in the two-dimensional image F(t1) or the like by forming the bright portion l, the dark portion d, and the boundary portion b. Each part of the inspection object T in F(t1)... can enter any one of the bright part l, the dark part d, and the boundary part b.

In addition, according to the present embodiment, the image integration data generated by dividing the image group G1(t1) by the column by the integration unit 11 is divided into the following pixels and output by the segmentation output unit 12, so that the image group G1 can be easily divided into columns. (t1), etc. Grasp the presentation mode of the bright part l, dark part d and boundary part b of the image of the combined image data, so that it is easy to further identify the category of the defect D of the inspection object T in the combined image data, wherein the aforementioned Each pixel is: it is a pixel of the image combination data obtained by photographing the same position of the inspection object T, and the brightness at the bright part l, the dark part d, and the boundary part b is above an arbitrary bright threshold value and is always bright A pixel; it is a pixel of image combination data obtained by photographing the same position of the inspection object T, and is always a dark pixel whose brightness at the bright part l, dark part d and boundary part b is below an arbitrary dark threshold; and It is the pixel of the combined image data obtained by photographing the same position of the inspection object T, and the change range of the brightness at the bright part l, the dark part d, and the boundary part b is rich in bright and dark changes above an arbitrary change threshold of pixels.

In addition, according to the present embodiment, the image integration data generated by dividing the image group G1(t1) etc. by the integration unit 11 into columns is divided into the coloring unit R for coloring the following pixels in different colors by the division output unit 12, G and B are output together, the above-mentioned each pixel refers to: it is the pixel of the image combination data obtained by photographing the same position of the inspection object T, and the brightness at the bright part l, dark part d and boundary part b is arbitrary. The pixels above the threshold are always bright pixels; they are the pixels of the image combination data obtained by shooting the same position of the inspection object T, and the brightness at the bright part l, dark part d and boundary part b is below an arbitrary dark threshold It is always a dim pixel; and it is a pixel of the image combination data obtained by shooting the same position of the inspection object T, and the magnitude of the brightness change at the bright part l, the dark part d and the boundary part b is an arbitrary change threshold The pixels above are rich in light and dark changes. In this way, by dividing the image group G1(t1) etc. into columns, the appearance of the bright part l, dark part d, and boundary part b of the image of the image combination data can be easily grasped by color, so it is easier to recognize the image. Class of defect D like inspection object T in merged data.

In addition, according to the present embodiment, even in the image merged data in which the pixel values of the image data are merged, because the image group G1(t1) etc. are divided and outputted by row, the image merged Since the representation form of the image of the document differs for each column segmented image group G1 ( t1 ), etc., it is easy to recognize the type of the defect D of the inspection target T in the integrated image document.

In addition, according to the present embodiment, even if the image data is the image data in which the values of the pixels of the image data are merged, the image group G1(t1) and the like are segmented by a predetermined threshold and output. , so that the presentation of the image of the image merged data is different according to the preset threshold between the column-divided image groups G1(t1), etc., so it is easy to identify the type of defect D of the inspection object T in the image merged data .

In addition, according to the present embodiment, for each column segmented image DL1 ( t1 ) etc. classified into the same column segmented image group G1 ( t1 ) etc., the combination unit 11 uses images obtained by photographing the same position of the inspection object T Based on the brightness of the pixels of the image data relative to the reference value, the pixels are given a difference value with a positive and negative sign, and the difference values of the pixels of the image data obtained by shooting the same position of the inspection object T are combined with each other. By dividing the image group G1(t1) etc. into columns to generate the combined image data C1(t1) etc., it is possible to divide the image group G1(t1) etc. Image integration data C1 ( t1 ) and the like in which the luminance of the pixels of the captured image data is enhanced relative to the reference value.

In addition, according to the present embodiment, for each column segmented image DL1 ( t1 ) classified into the same column segmented image group G1 ( t1 ) etc., the image obtained by capturing the same position of the test object T is enhanced. The enhancement processing of the brightness change of adjacent pixels of the image data is performed, and the values obtained by performing the enhancement processing of the pixels of the image data obtained by capturing the same position of the inspection object T are combined with each other, and the image group is divided into columns G1(t1) etc. generate image merged data C1(t1) etc., so the image group G1(t1) etc. can be divided into columns by simple calculations to generate image data obtained by capturing the same position of the inspection object T The image combination data C1(t1) and the like in which the luminance variation of pixels adjacent to each other are enhanced.

In addition, according to the present embodiment, based on the results of machine learning related to the recognition of the type of defects contained in the image integration data generated by dividing the image group G1(t1) etc. by the integration unit 11 into columns, the analysis unit 8 performs The accumulated data are used to identify the type of the defect D of the inspection object T, but the image data combined by combining the pixel values of the image data divides the image data by columns such as G1(t1) The values of the pixels are merged, so the type of the defect can be identified based on the result of machine learning for the image merged data C1(t1) etc., and the recognition accuracy of the type of the defect D of the inspection object T can be improved.

As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment, It can implement in various forms. For example, in the above-mentioned embodiment, the case where the inspection object T is a film has been mainly described, but the defect inspection system and defect inspection method of the present invention can be applied, for example, to inspection of the filling amount of liquid filled in containers in a production line. With the defect inspection system 1 and defect inspection method of this embodiment, it is possible to detect defects such as the liquid not reaching the desired position in the container or the liquid not exceeding the desired position in the container.

In addition, the defect inspection system 1 and the defect inspection method according to the present embodiment can be applied to visual inspections of fractures, flaws, and the like of glass products and the like in a production line. When glass products have defects such as cracks and scratches, defects can be extracted by utilizing the fact that the brightness is higher than other parts.

b: Boundary

D: defect

DL1, DLj, DLk: column segmentation image

d: Anbu

F: 2D image

L1, Lj, Lk: columns

l: bright department

t1, t2, t3, tm: time

X: conveying direction

Y: width direction

Claims (22)

A defect inspection system comprising: a light source for irradiating light to an inspection object; object or the aforementioned light reflected on the aforementioned inspection object; the conveying unit transports the aforementioned inspected object relative to the aforementioned light source and the aforementioned imaging unit along the conveying direction; and the image processing unit is configured by the aforementioned The image data of the two-dimensional image captured by the imaging unit is processed, the imaging unit captures the two-dimensional image whose brightness changes in a direction consistent with the conveying direction of the two-dimensional image, and the image The processing unit has: a column division processing unit that divides the two-dimensional image into a plurality of columns arranged along the conveying direction, and processes the two-dimensional image into the image data of the column-divided image, and the column The segmented image is formed by arranging the columns at the same position in each of the two-dimensional images captured by the imaging unit at the discrete time in a time-series order; Each of the aforementioned column segmented images processed by the processing unit is classified into two or more column segmented image groups according to preset rules; column segmentation image, the aforementioned inspection object The pixel values of the aforementioned image data obtained by shooting at the same position are merged with each other, and the image group is divided into the aforementioned row to generate the combined image data; The image merging data is divided and output according to the preset rules. The defect inspection system according to claim 1, wherein the two-dimensional image captured by the imaging unit in discrete time has a bright part, a dark part, and a boundary between the bright part and the dark part, and the boundary The portion is an intermediate portion between the portion with the highest brightness included in the bright portion and the portion with the smallest brightness included in the dark portion, and the conveying portion moves the inspection object relative to the imaging portion along the bright portion, the dark portion, and the boundary portion. The intersecting aforementioned conveying directions are oppositely conveyed. The defect inspection system according to claim 2, wherein the classifying unit classifies each of the row segmented images processed by the row segment processing unit into: the brighter part of each of the two-dimensional images The aforementioned column segmented image group of the aforementioned column segmented images arranged in the order of time series; the group of column-divided images; and the group of column-divided images of the column-divided images in which the columns of the boundary portions in each of the two-dimensional images are arranged in time-series order . The defect inspection system described in item 2 or item 3 of the patent application, wherein, The defect inspection system further includes a light-shielding body positioned between the light source and the inspection object for blocking a part of the light irradiated from the light source to the inspection object so that the image is captured by the imaging unit at discrete times. The aforementioned bright portion, the aforementioned dark portion, and the aforementioned boundary portion are formed on the aforementioned two-dimensional image. The aforementioned conveying directions that partially intersect are conveyed relatively. The defect inspection system described in Claim 2, wherein the segment output unit divides the image combination data generated by the combination unit by dividing the image group by the row into the following pixels and outputs them. It means: it is the aforementioned pixel of the aforementioned image combination data obtained by photographing the same position of the aforementioned inspection object, and is the aforementioned pixel whose luminance at the aforementioned bright portion, the aforementioned dark portion, and the aforementioned boundary portion is above any bright threshold value; The above-mentioned pixel of the above-mentioned image combination data obtained by shooting the same position of the above-mentioned inspection object is the above-mentioned pixel whose brightness at the above-mentioned bright part, the aforementioned dark part, and the aforementioned boundary part is below any dark threshold value; and it is the aforementioned pixel for the aforementioned The aforementioned pixels of the aforementioned combined image data obtained by photographing at the same position of the inspection object are the aforementioned pixels whose magnitude of change in luminance at the bright portion, the dark portion, and the boundary portion is greater than any change threshold. The defect inspection system as described in claim 5 of the patent claims, wherein the segment output unit divides the image combination data generated by the combination unit by dividing the image group by the row into a color for each of the pixels. The coloring parts of different colors are colored and output, and each of the above-mentioned pixels refers to the above-mentioned pixels of the above-mentioned image combination data obtained by photographing the same position of the above-mentioned inspection object and are the above-mentioned light part, the above-mentioned dark part and the above-mentioned The aforementioned pixel whose brightness at the boundary part is above any bright threshold; it is the aforementioned pixel of the aforementioned image combination data obtained by shooting the same position of the aforementioned inspection object, and it is the aforementioned pixel at the aforementioned bright part, the aforementioned dark part, and the aforementioned boundary part The above-mentioned pixel whose brightness is below an arbitrary dark threshold; and the above-mentioned pixel of the above-mentioned image combination data obtained by photographing the same position of the above-mentioned inspection object, and it is the change in brightness at the above-mentioned bright part, the above-mentioned dark part, and the above-mentioned boundary part The magnitude of any change threshold is above the preceding pixel. The defect inspection system according to claim 1, wherein the segment output unit divides and outputs the image combination data generated by the merging unit for each row segment image group by row segment image group. The defect inspection system described in claim 1 of the patent application, wherein the segment output unit divides the image combination data generated by the combination unit by dividing the image group by row according to the preset settings between the row segment image groups The threshold is segmented and output. The defect inspection system according to claim 1, wherein the merging unit, for each of the row segmented images classified into the same row segmented image group, is obtained by photographing the same position of the inspection object The luminance of the aforementioned pixel of the aforementioned image data is relative to the level of the reference value, so as to give the aforementioned pixel a value with a positive or negative sign. and combine the aforementioned difference values of the aforementioned pixels of the aforementioned image data obtained by photographing the same position of the aforementioned inspection object, and divide the image group by the aforementioned columns to generate the combined image data. The defect inspection system described in claim 1 of the patent application, wherein the merging unit performs enhancement on each of the row segmented images classified into the same row segmented image group by photographing the same position of the inspection object to obtain The enhanced processing of the brightness changes of the aforementioned pixels adjacent to each other in the aforementioned image data, and combining the values of the aforementioned enhanced processed pixels of the aforementioned image data obtained by shooting the same position of the aforementioned inspection object with each other, according to The aforementioned column segmentation of image groups generates image merge data. The defect inspection system according to claim 1, wherein the defect inspection system further includes an analysis unit based on a result of machine learning related to the recognition of the type of defects included in the image integration data The accumulated data is used to identify the type of defect of the inspection object, wherein the image combined data is generated by the combining unit by dividing the image group by the row. A defect inspection method comprising: an irradiation step of irradiating light from a light source of a defect inspection system to an inspection object; and an imaging step of capturing two-dimensional images at discrete times by an imaging unit of the defect inspection system, wherein the two-dimensional images It is based on the fact that in the aforementioned irradiation process, the aforementioned light source is irradiated to the aforementioned inspection object and penetrates the The object to be inspected or the light reflected on the object to be inspected; a conveying process in which the object to be inspected is relatively conveyed by the conveying unit of the defect inspection system along the conveying direction relative to the light source and the imaging unit; and An image processing step of processing the image data of the two-dimensional image captured in the imaging process by the image processing unit of the defect inspection system. For the aforementioned two-dimensional image whose brightness changes in a direction consistent with the aforementioned conveying direction, the aforementioned image processing step includes: dividing the aforementioned two-dimensional image into a plurality of columns arranged along the aforementioned conveying direction, and dividing the aforementioned Two-dimensional image processing A column division processing step of the image data of column-divided images, wherein the column-divided images are obtained by making each of the two-dimensional images captured by the imaging unit at the discrete time The aforementioned columns at the same position in the array are arranged in the order of time series; each of the aforementioned column segmented images processed in the aforementioned column segmenting process is classified into two or more column segmented image groups according to preset rules A classification process; for each of the aforementioned row segmented images classified into the same aforementioned row segmented image group in the aforementioned classifying step, the pixel values of the aforementioned image data obtained by photographing the same position of the aforementioned inspection object are combined with each other. And, the merging process of generating image merging data by dividing the image group by the aforementioned row; and the segmentation of dividing and outputting the aforementioned image merging data generated by dividing the image group by the aforementioned row in the aforementioned merging process according to preset rules output process. The defect inspection method according to claim 12, wherein, in the irradiation step, the two-dimensional image captured at discrete times in the imaging step has a bright part, a dark part, and the bright part and the aforementioned The boundary part of the dark part, the boundary part is the middle part between the part with the highest brightness included in the bright part and the part with the smallest brightness included in the dark part. The conveying directions in which the bright portion, the dark portion, and the boundary portion intersect are relatively conveyed. The defect inspection method according to claim 13, wherein, in the classification step, each of the row-segmented images processed in the row-segmentation processing step is classified into: each of the two-dimensional images The aforementioned column segmented image group of the aforementioned column segmented images formed by arranging the aforementioned columns of the aforementioned bright parts in the order of time series; the row segmented image group of the row segmented images arranged in a row; Columns split image groups. The defect inspection method described in claim 13 or claim 14, wherein, in the irradiation process, the light-shielding body is used to form the bright light on the two-dimensional image captured at discrete times in the imaging process. portion, the aforementioned dark portion, and the aforementioned boundary portion, the aforementioned light-shielding body is located between the light source and the inspection object, and shields part of the light irradiated from the aforementioned light source to the aforementioned inspection object. The light source, the light-shielding body, and the imaging unit are conveyed relatively along the conveying direction intersecting the bright portion, the dark portion, and the boundary portion. The defect inspection method according to claim 13 of the patent application, wherein, in the above-mentioned segmentation and output process, the aforementioned image integration data generated by dividing the image group by the aforementioned column in the aforementioned integration process is divided into the following pixels and processed Output, each of the above-mentioned pixels refers to: the above-mentioned pixel of the above-mentioned image combination data obtained by shooting the same position of the above-mentioned inspection object, and the brightness of the above-mentioned bright part, the aforementioned dark part, and the aforementioned boundary part is above an arbitrary bright threshold value The aforementioned pixel; it is the aforementioned pixel of the aforementioned image combination data obtained by shooting the same position of the aforementioned inspection object, and is the aforementioned pixel whose brightness at the aforementioned bright part, aforementioned dark part, and aforementioned boundary part is below any dark threshold; And it is the aforementioned pixel of the aforementioned image combination data obtained by photographing the same position of the aforementioned inspection object, and is the aforementioned pixel whose magnitude of change in brightness at the aforementioned bright portion, the aforementioned dark portion, and the aforementioned boundary portion is greater than or equal to an arbitrary change threshold . The defect inspection method described in claim 16 of the patent application, wherein, In the division and output step, the image integration data generated by dividing the image group by row in the integration step is divided into coloring parts in which the pixels are colored in different colors and output, and the pixels are Refers to: it is the aforementioned pixel of the aforementioned image combination data obtained by photographing the same position of the aforementioned inspection object, and the aforementioned pixel whose brightness at the aforementioned bright portion, the aforementioned dark portion, and the aforementioned boundary portion is above any bright threshold value; it is The above-mentioned pixels of the above-mentioned image combined data obtained by shooting the same position of the above-mentioned inspection object are the above-mentioned pixels whose brightness at the above-mentioned bright part, the above-mentioned dark part, and the above-mentioned boundary part is below an arbitrary dark threshold; The aforementioned pixels of the aforementioned image combination data obtained by shooting at the same position of the object are the aforementioned pixels whose brightness changes at the bright portion, the dark portion, and the boundary portion are greater than any change threshold. The defect inspection method according to claim 12 of the patent application, wherein, in the above-mentioned segmentation and output process, the image combination data generated by dividing the image group by the column in the above-mentioned combination process is divided into the image group by the column Split and output. The defect inspection method according to claim 12 of the patent application, wherein, in the segment output process, the image combination data generated by dividing the image group by row in the merge step is divided into the image group by row Between the preset thresholds are segmented and output. The defect inspection method according to claim 12, wherein, in the merging step, for each of the row segmented images classified into the same row segmented image group, based on the correlation of the inspection target The brightness of the aforementioned pixel of the aforementioned image data obtained by shooting at the same position relative to the reference value is used to assign a difference value with a positive or negative sign to the aforementioned pixel, and the aforementioned image data obtained by shooting the same position of the aforementioned inspection object The aforementioned difference values of the aforementioned pixels of the image data are combined with each other, and the image group is divided into the aforementioned columns to generate image combined data. The defect inspection method according to claim 12, wherein, in the merging step, for each of the row segmented images classified into the same row segmented image group, the same position of the inspection object is enhanced performing enhancement processing on the brightness changes of adjacent pixels of the image data obtained by photographing, and comparing the enhanced values of the pixels of the image data obtained by photographing the same position of the inspection object with each other Merge, divide the image group according to the aforementioned columns to generate image merge data. The defect inspection method described in claim 12 of the patent application, wherein the defect inspection method further includes an analysis process, and in the analysis process, based on the mechanical The data obtained by accumulating the result of the learning is used to identify the type of the defect of the inspection object, wherein the image combination data is the image combination data generated by dividing the image group by the row in the combination process.

Images