TWI788387B - Defect inspection system and defect inspection method - Google Patents

Abstract

The present invention provides a defect inspection system and a defect inspection method. In a defect inspection system (1), two-dimensional images F (t1) to F (tm) etc. whose brightness changes in a conveying direction (X) are imaged by an imaging unit (3), and the two-dimensional images F(t1) to F(tm) etc. are divided into lines L1 (t1) to Lk (tm) etc. arranged in parallel in the conveying direction (X) by a line division processing unit (9), and processed to become image data of line division images DL1 (t1) to DLk (t(1-(k-1)) which are obtained by arranging lines L1 (t1) to Lk (tm) etc. at the same position in the two-dimensional images F (t1) to F (tm) etc. in the order of time series, wherein the line division images DL1 (t1) etc. have different brightness even in the imaged images for the same inspection target (T). The kinds of defects (D) of the inspection target (T) are identified by a defect kind identifying unit (10) based on data accumulating the results of machine learning relating to the identification of the types of defects contained in the line division images DL1 (t1) to DLk (t(1-(k-1)) whose brightness and appearance are different, thereby improving identification accuracy.

Description

Defect inspection system and defect inspection method

The present invention relates to a defect inspection system and a defect inspection method.

As a defect inspection system that inspects defects of an inspection object based on captured images of the inspection object, for example, a defect inspection system that detects defects in optical films such as polarizing films and retardation films, laminated films used in battery separators, and the like is known. system. Such a defect inspection system transports a film along a transport direction, captures two-dimensional images of the film at discrete times, and performs defect inspection based on the captured two-dimensional images. For example, the system of Japanese Patent No. 4726983 generates a column segmented image by dividing a two-dimensional image into a plurality of columns arranged along the conveying direction, and making the two-dimensional images captured at discrete times Columns at the same position in each image are arranged in time-series order. The column-segmented image is processed into a defect-enhanced processed image in which brightness variations are enhanced. By processing the image with defect enhancement, the presence or absence of a film defect and the position of a film defect can be specified easily.

However, even if the two-dimensional image of the inspection object is processed into a defect-enhanced image as in the above-mentioned technology, the defect is finally identified based on human judgment, and there is still room for improvement in the accuracy of defect identification.

Therefore, an object of the present invention is to provide a defect inspection system and a defect inspection method capable of improving recognition accuracy of defects.

The present invention relates to a defect inspection system, comprising: a light source for irradiating light to an inspection object; It is formed by the light reflected on the inspection object; the conveying part is for conveying the inspection object relative to the light source and the imaging part along the conveying direction; and the image processing part is for the two-dimensional image captured by the imaging part The image data of the two-dimensional image is processed by the imaging unit, and the two-dimensional image whose brightness changes in the direction consistent with the conveying direction of the two-dimensional image is taken by the imaging unit. Processes the image data of the column-segmented image. The column-segmented image is obtained by dividing the two-dimensional image into a plurality of columns arranged along the conveying direction, and making the two-dimensional image captured by the imaging unit at discrete times Columns at the same position in each are arranged in chronological order; and the defect type identification unit is based on the result of machine learning related to the identification of the type of defects contained in the segmented images of two or more columns. The accumulated data are used to identify the type of defect of the inspected object, wherein the more than two row-segmented images are images obtained by processing the row-segmentation processing unit.

According to this configuration, the defect inspection system includes: a light source for irradiating light onto the inspection object; The object is formed by the light reflected on the object; the conveying part is to transport the inspection object relative to the light source and the imaging part along the conveying direction; The two-dimensional image whose brightness changes in the direction consistent with the transport direction of the two-dimensional image is captured by the imaging unit, and the two-dimensional image is processed by the column division processing unit of the image processing unit. Image data of a column-divided image, wherein the column-divided image is obtained by dividing a two-dimensional image into a plurality of columns arranged along the transport direction, and making the two-dimensional image captured by the imaging unit at discrete times The columns at the same position in each are arranged in a time-series order, and therefore, even in images obtained by capturing the same inspection object, the divided images of each column have different luminances. Furthermore, the defect type recognition part of the image processing part recognizes the defects of the inspection object based on the data obtained by accumulating the results of machine learning related to the recognition of the types of defects contained in the following series of segmented images category, wherein the row segmented image is two or more row segmented images with different luminances processed by the row segment processing unit. The classification of defects can be identified based on the results of machine learning performed on two or more column segmented images with different luminance and different presentation methods, thereby improving the recognition accuracy of defects.

In this case, it is preferable that the defect type identification unit based on the accumulated data of the results of machine learning related to the identification of the type of defects contained in two or more row segmented images different in brightness by 10% or more Identifies the type of defect of the inspected item.

According to this configuration, the defect type recognition unit recognizes the defect of the inspection object based on the accumulated data of machine learning results related to the recognition of the types of defects included in the two column segmented images with different luminances of 10% or more. Therefore, even in images taken of the same inspection object, defects are identified based on the results of machine learning for two column-segmented images whose brightness differs by more than 10% and whose presentation differs greatly category, so the recognition accuracy of defects can be further improved.

In addition, it is preferable that the defect inspection system further includes a light-shielding body that is positioned between the light source and the inspection object and that shields part of the light irradiated from the light source to the inspection object, so that the two images captured by the imaging unit at discrete times The bright part and the dark part are formed on the dimensional image, and the conveying part relatively transports the inspected object relative to the light source, light-shielding body and imaging part along the conveying direction intersecting the boundary line between the bright part and the dark part. The data obtained by accumulating the results of machine learning related to the recognition of the type of defect contained in the segmented image is used to identify the type of defect in the inspection object. The segmented image refers to: making the bright part of the two-dimensional image The column-segmented image in which the columns of the positions of the dark parts are arranged in the order of time series; and the column-segmented image in which the columns of the positions of the dark parts in the two-dimensional image are arranged in the order of time series.

According to this configuration, a part of the light irradiated from the light source to the inspection object is blocked by the light shielding body positioned between the light source and the inspection object, thereby forming bright and dark portions on the two-dimensional image captured by the imaging unit at discrete times, The inspection object is transported by the transport unit relative to the transport direction that intersects with the light source, light-shielding body and imaging unit along the boundary line between the bright part and the dark part, so in a series of two-dimensional images captured at discrete times Check that each part of the object enters both the bright part and the dark part. In addition, the defect type recognition unit recognizes the type of defect of the inspection object based on the data obtained by accumulating the results of machine learning related to the recognition of the type of defects included in the row segmented image, wherein the row segmented image The image refers to: a column division image formed by arranging the columns of the positions of the bright parts in the two-dimensional image in the order of time series; Sequentially arranged column segmentation images, so based on the results of machine learning for two column segmentation images that belong to the bright part and the dark part and have greatly different presentation methods, the type of defect can be identified, which can further improve the accuracy of defects. recognition accuracy.

In another aspect, the present invention is a defect inspection method including: an irradiation step of irradiating light from a light source of a defect inspection system to an inspection object; and an imaging step of capturing two-dimensional images at discrete times by an imaging unit of the defect inspection system, wherein , the two-dimensional image is formed based on the light irradiated from the light source to the inspection object in the irradiation process and passed through the inspection object or reflected on the inspection object; The conveying process in which the conveying direction is oppositely transported; and the image processing process in which the image data of the two-dimensional image captured in the imaging process is processed by the image processing unit of the defect inspection system. In the imaging process, the captured The two-dimensional image whose luminance changes in the direction coincident with the conveyance direction of the two-dimensional image includes a row division processing step of processing the two-dimensional image into image data of a row-divided image in the image processing step , wherein the column division image is divided into a plurality of columns arranged along the transport direction by dividing the two-dimensional image, and making the columns of the same position in each of the two-dimensional images captured at discrete times in the imaging process Arranged according to the order of time series; and based on the accumulated data of the results of machine learning related to the recognition of the types of defects contained in two or more series of segmented images, to identify the type of defects in the inspection object The defect type recognition step of the above, wherein the two or more column-segmented images are images processed in the column-segmentation processing step.

In this case, it is preferable that in the defect type identification process, based on the accumulated data obtained based on the results of machine learning related to the identification of the type of defects included in the two column segmented images that differ in brightness by 10% or more, To identify the type of defect of the inspected object.

In addition, it is preferable that in the irradiation process, bright and dark portions are formed on the two-dimensional image captured at discrete times in the imaging process by a light-shielding body located between the light source and the inspection object, and the light-shielding body is located between the light source and the object to be inspected. Part of the light irradiated by the inspection object is blocked. In the conveying process, the inspection object is relatively conveyed relative to the light source, light-shielding body, and imaging unit along the conveying direction intersecting the boundary line between the bright part and the dark part. In the defect type identification process In this method, the types of defects in the inspection object are identified based on data accumulated as a result of machine learning related to the identification of the types of defects included in the following row segmented images by using A column segmentation image in which the columns of the positions of the bright parts in the two-dimensional image are arranged in the order of time series; Column split image.

1‧‧‧Defect inspection system

2‧‧‧Light source

3: Camera Department

4: Conveying department

5: Image processing department

6: Shading body

7: parallel light lens

8: Display device

9: Column division processing part

10: Defect category identification department

100: Convolutional Neural Networks

110: Input layer

120: hidden layer

121: Convolution layer

122:Pooling layer

123: Convolution layer

124: Fully connected layer

130: output layer

b: dividing line

D: defect

d: Anbu

F: 2D image

l: bright department

R: error to reverse

T: check object

X: conveying direction

Y: width direction

DL1, DLj, DLk: column segmentation image

L1, Lj, Lk: columns

S1: Irradiation process

S2: Camera process

S3: Conveying process

S4: Image processing process

S41: Column division processing step

S42: Defect category identification process

t, t1, t2, t3, tm: time

Fig. 1 is a perspective view showing a defect inspection system according to an embodiment.

Fig. 2 is a diagram showing the arrangement of a light source, an imaging unit, a light shield, and an inspection object in the defect inspection system of Fig. 1 .

FIG. 3 is a block diagram showing details of an image processing unit of the defect inspection system in FIG. 1 .

Fig. 4 is a flow chart showing the steps of the defect inspection method of the embodiment.

FIG. 5 is a flow chart showing details of the image processing process in FIG. 4 .

(A), (B), (C), (D), (E), (F) and (G) in Fig. 6 show the column division processing performed by the image processing unit of the defect inspection system in Fig. 1 A plot of the partially processed image.

(A) of Fig. 7 is a diagram showing a time-series two-dimensional image, (B) is a diagram showing a divided image of each column in which the columns of each position are arranged in the order of time series, and (C) is It shows a diagram of a bitmap image obtained by shifting time so that each row of divided images in (B) shows the same position of the inspection object.

Figure 8 is a diagram showing a convolutional neural network.

Hereinafter, preferred embodiments of the defect inspection system and defect inspection method of the present invention will be described in detail with reference to the drawings. As shown in FIGS. 1 and 2, a defect inspection system 1 according to an embodiment of the present invention includes a light source 2, an imaging unit 3, a transport unit 4, an image processing unit 5, a light shielding body 6, a parallel light lens 7, and a display device. 8. The defect inspection system of this embodiment uses optical films such as polarizing films and retardation films, and laminated films used for battery separators as inspection objects T, and detects defects in the inspection objects T. The inspection object T extends along the conveying direction X of the conveying unit 4 , and has a predetermined width in the width direction Y perpendicular to the conveying direction X. Defects generated in the inspection object T refer to a state different from the expected state, such as foreign matter, scratches, bubbles (bubbles generated during molding, etc.), foreign matter bubbles (bubbles generated by the mixing of foreign matter, etc.) ), scratches, cracks (cracks due to creases, etc.), and streaks (streaks due to differences in thickness, etc.). The defect inspection system 1 recognizes the categories of these defects. The defect inspection system 1 can identify on which side of the inspection object T the defect occurred, in addition to identifying the type of the defect.

As shown in FIGS. 1 and 2 , the light source 2 irradiates the inspection object T with light. The light source 2 is arranged so as to emit linear light parallel to the width direction Y. The light source 2 is not particularly limited as long as it is a lamp such as a metal halide lamp, a halogen transmission lamp, or a fluorescent lamp that emits light that does not affect the composition and properties of the film of the inspection object T.

The imaging unit 3 captures a two-dimensional image formed based on light irradiated from the light source 2 to the inspection object T and transmitted through the inspection object T or reflected on the inspection object T at discrete times. The imaging unit 3 has a plurality of optical members and photoelectric conversion elements. The optical member includes an optical lens, a shutter, and the like, and forms an image of light transmitted through a film as the inspection object T on the surface of the photoelectric conversion element. The photoelectric conversion element is a CCD (Charge Coupled Device, Charge Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor, Complementary Metal Oxide Semiconductor) that captures a two-dimensional image. Surface sensor composed of elements. The imaging unit 3 may be a member that captures either a two-dimensional image without color or a two-dimensional image with color.

The transport unit 4 transports the inspection object T relative to the light source 2 and the imaging unit 3 along the transport direction X. The transport unit 4 includes, for example, a sending roller and a receiving roller that transport the film as the inspection object T along the transport direction X, and measures the transport distance with a rotary encoder or the like. In the present embodiment, the conveying speed of the conveying unit 4 conveying the inspection object T is set to about 2 to 100 m/min along the conveying direction X. The transport speed of the transport unit 4 is set and controlled by the image processing unit 5 and the like.

The image processing unit 5 processes the image data of the two-dimensional image captured by the imaging unit 3 . The image processing unit 5 is not particularly limited as long as it is a member that performs image processing of two-dimensional image data, and for example, a PC (personal computer) equipped with image processing software, or an FPGA equipped with an image processing circuit can be applied. (Field Programmable Gate Array, Field Programmable Gate Array) image acquisition card, etc.

The light-shielding body 6 is located between the light source 2 and the inspection object T, and blocks a part of the light irradiated from the light source 2 to the inspection object T to form a bright portion on the two-dimensional image captured by the imaging unit 3 at discrete times. and Anbe. Through the light shielding body 6 , the imaging unit 3 captures a two-dimensional image whose brightness changes in a direction that coincides with the conveying direction X of the two-dimensional image. More specifically, the transport unit 4 transports the inspection object T relative to the light source 2 , collimator lens 7 , light-shielding body 6 and imaging unit 3 along the transport direction X intersecting the boundary line between the bright part and the dark part. In this embodiment, the boundary line is parallel to the width direction Y perpendicular to the conveyance direction X. In addition, as long as the imaging unit 3 can capture a two-dimensional image whose brightness changes in a direction that coincides with the transport direction X of the two-dimensional image, the light shielding body 6 does not need to be provided. The parallel light lens 7 parallelizes the traveling direction of light irradiated from the light source 2 to the inspection object T and the light shielding body 6 . The parallel light lens 7 can be constituted by a telecentric optical system, for example.

The display device 8 connected to the image processing unit 5 is composed of, for example, a PC (personal computer), and displays the type of the defect recognized by the image processing unit 5 on an LC (Liquid Crystal, liquid crystal) display panel, plasma display panel, EL (Electro Luminescence, electroluminescent) display panel, etc. In addition, the image processing unit 5 may include a display device for displaying the processed image.

Hereinafter, details of the image processing unit 5 will be described. As shown in FIG. 3 , the image processing unit 5 has a column division processing unit 9 and a defect type recognition unit 10 . The column division processing section 9 processes the two-dimensional image into image data of a column-divided image obtained by dividing the two-dimensional image into a plurality of columns arranged along the transport direction X, and making The two-dimensional images captured by the imaging unit 3 at discrete times are those in which columns at the same position are arranged in time-series order. The defect type recognition unit 10 recognizes the inspection object T based on the accumulated data of machine learning results related to the recognition of the types of defects included in the two or more row segmented images processed by the row segment processing unit 9 . category of defects. The data which accumulated the result of machine learning is memorize|stored in storage devices, such as a hard disk of PC which includes the defect type identification part 10, and is updated with the result of machine learning.

It should be noted that, in this embodiment, the data obtained by accumulating the results of machine learning related to the recognition of the types of defects included in the row segmented images includes, in addition to In addition to the data obtained by accumulating the results of machine learning related to the identification of the types of defects contained in the series of divided images captured by the imaging unit 3 at discrete times and processed, it also includes data related to the The data obtained by accumulating the results of machine learning related to the recognition of the types of defects included in the column segmented images separately generated outside the defect inspection system 1 . That is, in this embodiment, in addition to the scheme of identifying the type of defect in the state where machine learning has been performed in the defect inspection system 1, it also includes a method based on the state where machine learning has not been performed in the defect inspection system 1 . It is a plan to identify the type of defect by accumulating the data obtained by accumulating the result of machine learning separately generated outside the defect inspection system 1 .

The defect type recognition unit 10 recognizes the type of defect in the inspection object T based on accumulated data obtained from the results of machine learning related to the recognition of the types of defects included in the two column segmented images whose luminance is different by 10% or more. . In addition, the defect type recognition unit 10 recognizes the type of defects in the inspection object T based on the accumulated data of machine learning results related to the recognition of the types of defects included in the sequence of segmented images. The image refers to: a column division image formed by arranging the columns of the positions of the bright parts in the two-dimensional image obtained by means of the light-shielding body 6 in the order of time series; A column-segmented image in which the columns of the positions of the dark parts in the two-dimensional image are arranged in the order of time series.

Hereinafter, the defect inspection method of this embodiment will be described. As shown in FIG. 4 , an irradiation step ( S1 ) of irradiating light from the light source 2 of the defect inspection system 1 to the inspection object T is performed. As shown in (A) of FIG. 6 , in the irradiation process, the light-shielding body of the defect inspection system 1 that is located between the light source 2 and the inspection object T and blocks part of the light irradiated from the light source 2 to the inspection object T is used. 6. Form a bright portion l and a dark portion d separated by the boundary line b on the two-dimensional image F(t1) captured at discrete times during the imaging process. As shown in (A) of Fig. 6, for the two-dimensional image F(t1) at time t=t1, the light from the light source 2 is blocked by the light shielding body 6, so as it reaches the downstream side of the transport direction X On the other hand, the brightness in the two-dimensional image F(t1) becomes high. In addition, the defect D on the film of the inspection object T is projected on the two-dimensional image F(t1). The same applies to the two-dimensional images F(t2), F(t3), ..., F(tm) at time t=t2, t3, ..., tm (m is an arbitrary natural number).

As shown in FIG. 4, the imaging process (S2) is performed by the imaging unit 3 of the defect inspection system 1. In this imaging process, a two-dimensional image F(t1) is captured at a discrete time. The two-dimensional image F(t1) ) is formed based on light irradiated from the light source 2 to the inspection object T and transmitted through the inspection object T or reflected on the inspection object T in the irradiation step. As shown in (A) of FIG. 6 , in the imaging process, part of the light irradiated from the light source 2 to the inspection object T is blocked by the light shielding body 6 , so that the two-dimensional image F(t1) and the transport direction are captured. Two-dimensional image F(t1) whose brightness changes in the same direction as X. The same applies to the two-dimensional images F(t2), F(t3), . . . , F(tm) at time t=t2, t3...tm.

In addition, as shown in FIG. 4 , the conveying process ( S3 ) of relatively conveying the inspection object T relative to the light source 2 and the imaging unit 3 along the conveying direction X is performed by the conveying unit 4 of the defect inspection system 1 . As shown in (A) of FIG. 6, in the conveying process, the inspection object T is placed along the boundary line b between the bright part l and the dark part d with respect to the light source 2, the collimator lens 7, the light shielding body 6, and the imaging part 3. The intersecting conveying directions X convey oppositely. In this embodiment, the boundary line b is parallel to the width direction Y perpendicular to the conveyance direction X, but the angle formed between the boundary line b and the conveyance direction X may be an angle other than 90°. In addition, the boundary line b is not necessarily a strict boundary line, and the boundary line b is the part where the brightness of the two-dimensional image F(t1) included in the specified part l is the highest and the brightness of the two-dimensional image F included in the dark part d is the smallest. The middle line of the part.

k)。列L1(t1)~列Lk(t1)的輸送方向X的寬度與在時刻t1、時刻t2、…、時刻tj、…、時刻tm中的各時刻下的一幀間隔中將檢查物件T沿著輸送方向X輸送的距離相同。對時刻t=t2、t3…tm下的二維圖像F(t2)、F(t3)、…、F(tm)也進行同樣的處理。 As shown in FIG. 4, image processing for processing image data of two-dimensional images F(t1) to F(tm) captured in the imaging process is performed by the image processing unit 5 of the defect inspection system 1. Process (S4). Hereinafter, details of the image processing step will be described. As shown in FIG. 5 , in the image processing step, the column division processing unit 9 of the image processing unit 5 of the defect inspection system 1 performs a column division processing step ( S41 ). As shown in (B) of FIG. 6 , in the column division processing step, the column division processing unit 9 divides the two-dimensional image F(t1) into a plurality of first columns L1(t1) arranged along the transport direction X. )~jth column Lj(t1)~kth column Lk(t1) (j and k are any natural numbers, j

k). The width of the conveying direction X of the column L1(t1)~Lk(t1) is the same as the width of the inspection object T along the frame interval at each time of time t1, time t2, ..., time tj, ..., time tm The distance conveyed in the conveying direction X is the same. The same processing is performed on the two-dimensional images F(t2), F(t3), . . . , F(tm) at time t=t2, t3...tm.

The column division processing unit 9 processes the two-dimensional images F(t1)~F(tm) into image data of column division images. Columns L1 ( t1 ), L1 ( t2 ), etc. at the same position in each of the images F( t1 )˜F(tm) are arranged in the order of time series. An example of the segmented image in the first column will be described. As shown in (C) of FIG. 6 , the column division processing unit 9 makes the two-dimensional images F(t1), F(t2), F(t3), ... each of the two-dimensional images captured at discrete times in the transport direction X The first row L1 ( t1 ), L1 ( t2 ), L1 ( t3 ), . . . on the most downstream side are arranged in chronological order (transportation direction X). As shown in (D) of FIG. 6 , the column division processing unit 9 arranges the first columns L1 ( t1 ) to L1 ( tm ) in each of the two-dimensional images F ( t1 ) to F ( tm ) in order of time series. The first column of divided images DL1 ( t1 ) is generated by arranging them.

As shown in (E) of FIG. 6, (F) of FIG. 6, and (G) of FIG. The first column L1(t1)~L1(tm),..., the jth column Lj(t1)~Lj(tm),..., the kth column Lk(t1)~Lk(tm) perform the same processing to generate the first Column divided image DL1(t1), ..., j-th column divided image DLj(t1), ..., k-th column divided image DLk(t1). As shown in (E) of Fig. 6, the row segmented image DL1(t1) is obtained by making the row L1(t1)~ L1(tm) is a column segmentation image arranged in the order of time series. In addition, as shown in (F) of FIG. 6 , the row division image DLj(t1) is obtained by making the row Lj at the position near the boundary line b in the two-dimensional images F(t1)~F(tm) (t1)~L1(tm) is a column segmentation image arranged in the order of time series. In addition, as shown in (G) of FIG. 6 , the row division image DLk(t1) is obtained by making the row Lk(t1) of the position of the dark part d in the two-dimensional images F(t1)~F(tm) ~Lk(tm) is a column segmentation image arranged in the order of time series.

As shown in (E) to (G) of Figure 6, the column division images DL1(t1)~DLk(t1) are two-dimensional images F(t1)~F taken at discrete times (tm) The column division images of columns L1(t1)~Lk(t1) at the same position in each of them are arranged in the order of time series, so the column division images DL1(t1)~ DLk(t1) represents different positions of the inspection object T, and the positions of the defects D in the row segmented images DL1(t1)˜DLk(t1) are also shifted respectively. Therefore, in this embodiment, by creating a column division image in which the columns at the same position in the two-dimensional images captured at different time ranges are arranged in the order of time series, the The divided images of each row are aligned so that the same position of the inspection object T is displayed.

As shown in (A) of FIG. 7 , in the imaging process, two-dimensional images F(t1) to F(tm) are captured at discrete times. Since the inspection object T is transported along the transport direction X, the positions of the defects D in the two-dimensional images F(t1)˜F(tm) are shifted respectively. As shown in (B) of FIG. 7 , the column divided images DL1 ( t1 ) to DLj ( t1 ) to DLk ( t1 ) are generated as described above. The row segmented images DL1(t1)~DLk(t1) in the range of the same time show different positions of the inspection object T, so the positions of the defects D in the row segmented images DL1(t1)~DLk(t1) are also respectively offset.

With respect to the first row L1(t1)~L1(tm) from the downstream side of the conveying direction X, for example, the jth row Lj(t1)~Lj( tm) shows a position shifted upstream in the conveyance direction X of the inspection object T by the distance the inspection object T is conveyed at a frame interval equal to (j-1). Therefore, as shown in (C) of FIG. 7, with respect to the column division image DL1(tm) of the first column L1(tm)~L1(t(m+(m-1))), for example, the j-th column For the column division image, it is time t(m-(j-1))~time t(m+( The column segmented image DLj(t(m-(j-1))) in the range of m-j)) will show the same position of the inspection object T.

Similarly, for the column division image DL1(tm) of the first column L1(tm)~L1(t(m+(m-1))), for example, for the column division image of the k-th column, relative to The column division image DLk in the range from time t(m-(k-1)) to time t(m+(m-k)) at which the frame interval of (k-1) is traced back in the range from time t1 to time tm (t(m-(k-1))) will show the same position of inspected object T.

Alternatively, with respect to the column division image DL1(t1) of the first column L1(t1)~L1(t(1+(m-1))), for example, for the column division image of the jth column, time t The column division image DLj(t(1-(j-1))) in the range from (1-(j-1)) to time t(1+(m-j)) shows the same position of the inspection object T. In addition, with respect to the column divided image DL1(t1) of the first column L1(t1)~L1(t(1+(m-1))), for example, for the column divided image of the k-th column, time t The column segmented image DLk(t(1-(k-1))) ranging from (1-(k-1)) to time t(1+(m-k)) shows the same position of the inspection object T. By shifting the range of time in this way, alignment can be performed so that the same position of the inspection object T is displayed in each row of divided images.

Also, when the amount of positional shift is known or the size of the column segmented image is sufficiently large for the defect, the defect will definitely fall into the column segmented image, so even if alignment is not performed, Ability to use column-segmented images containing defects for machine learning. Therefore, in such a case, alignment need not be performed.

As shown in FIG. 5, the defect type recognition process is performed by the defect type recognition part 10 of the image processing part 5 of the defect inspection system 1 (S42). In the defect type recognition step, the defect type recognition unit 10 recognizes the defect of the inspection object T based on the accumulated data of machine learning results related to the recognition of the type of defects contained in two or more row segmented images. In the category of D, the two or more column division images are two or more column division images DL1(t1), ..., DLj(t(1-(j-1)) processed in the column division processing step ),...,DLk(t(1-(k-1))).

In the defect type recognition step, the defect type recognition unit 10 is based on the machine learning related to the recognition of the types of defects contained in the two column division images DL1(t1) and DLk(t1) whose brightness differs by more than 10%. As a result, the accumulated data are used to identify the category of the defect D of the inspected object T. More specifically, in the defect type recognition step, the defect type recognition unit 10 recognizes the inspection object based on data accumulated as a result of machine learning related to the recognition of the types of defects included in the column segmented images as follows: The category of the defect D of T, the aforementioned column division image refers to: by making the columns L1(t1)~L1(tk) of the positions of the bright part 1 in the two-dimensional images F(t1)~F(tm) according to The column segmentation image DL1(t1) arranged in the order of time series; and by making the two-dimensional image F(t(1-(k-1)))~F(t(1+(m-k))) The columns Lk(t(1-(k-1)))~Lk(t(1+(m-k))) of the position of the dark part d in the column are arranged in the order of the time series and the column segmentation image DLk(t( 1-(k-1))). Machine learning is performed, for example, by convolutional neural networks. In addition, neural networks or other methods other than convolutional neural networks may be used as long as the types of defects can be recognized by machine learning.

As shown in FIG. 8 , the convolutional neural network 100 includes an input layer 110 , a hidden layer 120 and an output layer 130 . The image processing unit 5 of the defect inspection system 1 divides two or more columns in the column division images DL1(t1)~DLk(t(1-(k-1))) processed in the column division processing step The divided image is input to the input layer 110 . The hidden layer 120 includes: convolutional layers 121 and 123 for performing image processing based on weight filters; pooling layer 122 for processing to vertically and horizontally reduce the two-dimensional arrays output from the convolutional layers 121 and 123 to leave effective values; And update the fully connected layer 124 of the weight coefficient n of each layer. In the output layer 130 , the recognition result of the category of the defect D by machine learning is output. In the convolutional neural network 100 , the error between the output recognition result and the positive solution value is backpropagated to the reverse direction R to learn the weight of each layer.

For example, a plurality of column segmented images are input in advance to the image processing unit 5 together with a positive solution for identifying the type of the defect D to allow the image processing unit 5 to perform learning, thereby sequentially recognizing the newly input column segmented images DL1( t1) and so on whether the category included is a specific defect D category, and sequentially output the recognition results. The recognition results output sequentially and the error of the positive solution are reversely propagated to the reverse direction R, and the weight coefficient n of each layer is sequentially updated and accumulated as data. In the state where the weights of each phase are sequentially updated, it is further sequentially recognized whether the category contained in the newly input column segmented image DL1(t1) etc. is a specific defect category, and the recognition results are sequentially output. The weight coefficient n of each layer is sequentially updated according to the error between the result and the correct solution and accumulated as data. By repeating this process, the error between the recognition result and the correct solution is reduced, and the recognition accuracy of the defect D category is improved.

According to the present embodiment, the defect inspection system 1 includes: a light source 2 for irradiating light to the inspection object T; The images F(t1)~F(tm) are formed based on the light irradiated from the light source 2 to the inspection object T and transmitted through the inspection object T or reflected on the inspection object T; 2 and the imaging unit 3 are relatively transported along the transport direction X; and the image processing unit 5 is used to process image data such as two-dimensional images F(t1)~F(tm) taken by the imaging unit 3, wherein Two-dimensional images F(t1)-F(tm), etc. in which brightness changes in a direction consistent with the conveying direction X, such as two-dimensional images F(t1)-F(tm), etc., are captured by the imaging unit 3, And the two-dimensional images F(t1)~F(tm) etc. are processed into column division images DL1(t1)~DLk(t(1-(k-1) )) of the image data, the row segmentation image DL1(t1)~DLk(t(1-(k-1))) divides the two-dimensional image F(t1)~F(tm) along the A plurality of columns L1(t1)~Lk(tm) etc. arranged in the conveying direction X, and the same position in each of the two-dimensional images F(t1)~F(tm) etc. captured by the imaging unit 3 at discrete times The columns L1(t1)~L1(tm) are arranged in the order of time series, so even if it is an image obtained by shooting the same inspection object, each row of divided images DL1(t1)~DLk(t( 1-(k-1))) also becomes an image with different brightness. Furthermore, the defect type recognition unit 10 of the image processing unit 5 recognizes the defect of the inspection object T based on the accumulated data obtained from the results of machine learning related to the recognition of the type of defects included in the sequence of segmented images as follows: The category of D, wherein the column division images are two or more column division images DL1(t1)~DLk(t(1-(k-1 ))), therefore, even if it is an image taken of the same inspection object T, it will be based on two or more column division images DL1(t1)~DLk(t(1 -(k-1))) and so on to identify the type of defect D, so that the recognition accuracy of defect D can be improved.

In addition, according to the present embodiment, the defect type recognition unit 10 recognizes the defect of the inspection object T based on the data obtained by accumulating the results of machine learning related to the recognition of the type of defects included in the following two series of divided images. The category of D, wherein the two column segmented images are two column segmented images DL1(t1) and DLk(t(1-(k-1))) whose brightness is more than 10% different, therefore, even The image obtained by shooting the same inspection object T is also based on two column division images DL1(t1), DLk(t(1-(k- 1))) to identify the category of the defect D based on the result of the machine learning performed, so that the recognition accuracy of the defect D can be further improved.

In addition, according to the present embodiment, part of the light irradiated from the light source 2 to the inspection object T is blocked by the light shielding body 6 positioned between the light source 2 and the inspection object T. A bright part l and a dark part d are formed on the three-dimensional images F(t1)~F(tm), etc., and the inspection object T is moved by the conveying part 4 along the bright part l and the dark part relative to the light source 2, the light-shielding body 6 and the imaging part 3. The conveying direction X where the boundary line b of d intersects is relatively conveyed, so each part of the inspection object T in a series of two-dimensional images F(t1)~F(tm) captured at discrete times enters the bright part l And the two sides of the dark part d. In addition, the defect type recognition unit 10 recognizes the type of the defect D of the inspection object T based on the data obtained by accumulating the results of machine learning related to the recognition of the types of defects contained in the following row segmented images. The row segmented image is formed by arranging the rows L1(t1)~L1(tm) of the positions of the bright part 1 in the two-dimensional image F(t1)~F(tm) etc. in the order of time series The row segmentation image DL1(t1); and by making the position of the dark part d in the two-dimensional image F(t(1-(k-1)))~F(t(1+(m-k))) Column segmentation image DLk(t(1-(k-1) )), therefore, it becomes a machine learning based on two column segmentation images DL1(t1), DLk(t(1-(k-1))) which belong to the bright part l and the dark part d respectively and have greatly different presentation methods As a result, the category of the defect D can be identified, thereby further improving the identification accuracy of the defect D.

As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment, It can implement in various forms. For example, in the above-mentioned embodiment, the case where the inspection object T is a film has been mainly described, but the defect inspection system and defect inspection method of the present invention can be applied to, for example, inspection of the filling amount of liquid filled in containers in a production line. With the defect inspection system 1 and the defect inspection method of this embodiment, it is possible to detect defects such as the liquid not reaching the desired position in the container, or detecting that the liquid does not exceed the desired position in the container.

In addition, the defect inspection system 1 and the defect inspection method according to the present embodiment can be applied to visual inspections of fractures, flaws, and the like of glass products and the like in a production line. In the case of defects such as breakage and scratches in glass products, defects can be screened out by using the fact that the brightness is higher than other parts.

1‧‧‧Defect inspection system

2‧‧‧Light source

3‧‧‧camera department

4‧‧‧Transportation Department

5‧‧‧Image Processing Department

6‧‧‧Shading body

7‧‧‧Parallel light lens

T‧‧‧Check Object

X‧‧‧Transportation direction

Y‧‧‧width direction

Claims (6)

A defect inspection system comprising: a light source for irradiating light to an inspection object; an imaging unit for taking two-dimensional images at discrete times, the two-dimensional image being irradiated from the light source to the inspection object and passing through the inspection object or It is formed by the aforementioned light reflected on the aforementioned inspection object; the transport unit is for transporting the aforementioned inspection object relative to the aforementioned light source and the aforementioned imaging unit along the transport direction; and the image processing unit is for capturing images taken by the aforementioned imaging unit. The image data of the aforementioned two-dimensional image is processed, and the aforementioned imaging unit captures the aforementioned two-dimensional image whose brightness changes in a direction consistent with the aforementioned conveying direction of the aforementioned two-dimensional image, and the aforementioned image processing unit has : a row division processing unit, which processes the aforementioned two-dimensional image into the aforementioned image data of a row-divided image, wherein the aforementioned row-divided image is obtained by dividing the aforementioned two-dimensional image into a plurality of arrays arranged along the aforementioned conveying direction. row, which is formed by arranging the row at the same position in each of the two-dimensional images captured by the imaging unit at the discrete time in a time-sequential order, and the row division processing unit is obtained by making the aforementioned In each of the row divided images, the time ranges of the rows arranged in a chronological order are staggered so that each of the row divided images displays the same position of the inspection object; and a defect type identification unit , based on machine learning associated with the recognition of classes of defects contained in two or more preceding columns of segmented images The data obtained by accumulating the result to identify the type of the defect of the inspection object, wherein, the two or more row segmented images are images obtained by processing the row segment processing unit. The defect inspection system according to claim 1, wherein the defect type identification unit is based on machine learning related to the identification of the type of defects included in the two column segmented images that differ in brightness by 10% or more The data obtained by accumulating the results are used to identify the types of defects of the aforementioned inspected objects. The defect inspection system according to claim 1 or 2 of the patent claims, wherein the defect inspection system further includes a light-shielding body, which is located between the light source and the inspection object, and illuminates the light from the light source to the inspection object. Part of the aforementioned light is blocked to form a bright part and a dark part on the aforementioned two-dimensional image captured by the aforementioned imaging unit at a discrete time. Relatively conveying along the conveying direction intersecting the boundary line between the bright portion and the dark portion, the defect type identification unit performs based on the results of machine learning related to the identification of the type of defects contained in the following column segmented images The accumulated data are used to identify the type of defect of the inspection object. The row segmented image refers to a row segmented image obtained by arranging the rows of the positions of the bright parts in the two-dimensional image in a time series order. image; and make the position of the aforementioned dark part in the aforementioned two-dimensional image The aforementioned columns are arranged in the order of time series to divide the image into columns. A defect inspection method comprising: an irradiation step of irradiating light from a light source of a defect inspection system to an inspection object; and an imaging step of capturing a two-dimensional image at discrete times by an imaging unit of the defect inspection system, wherein the two-dimensional image Formed based on the light irradiated from the light source to the inspection object in the irradiation process and transmitted through the inspection object or reflected on the inspection object; a conveying step in which the imaging unit relatively conveys along the conveying direction; and an image processing step in which the image data of the two-dimensional image captured in the imaging step is processed by the image processing unit of the defect inspection system , in the aforementioned imaging step, the aforementioned two-dimensional image whose brightness changes in a direction consistent with the aforementioned conveying direction of the aforementioned two-dimensional image is photographed, and the aforementioned image processing step includes: processing the aforementioned two-dimensional image The column division processing step of the image data of the column-divided image, wherein the column-divided image is obtained by dividing the aforementioned two-dimensional image into a plurality of columns arranged along the aforementioned transport direction, and making In the process, the above-mentioned columns of the same position in each of the above-mentioned two-dimensional images captured at the above-mentioned discrete time are arranged in the order of time series, and the process of dividing the columns is by using The time ranges of the aforementioned columns in each of the aforementioned column segmented images are staggered according to the order of time series, so that each of the aforementioned column segmented images displays the same position of the aforementioned inspection object; and based on the alignment and The process of identifying the type of defect in the inspection object by accumulating the data obtained by accumulating the results of machine learning related to the recognition of the type of defect included in the two or more row segmented images, wherein two or more The column segmented image is an image processed in the column segment processing step. The defect inspection method according to claim 4 of the patent application, wherein, in the defect type identification step, based on the identification correlation of the types of defects contained in the two aforementioned column segmented images that differ by more than 10% in brightness The data obtained by accumulating the result of machine learning to identify the type of defect of the aforementioned inspected object. The defect inspection method according to claim 4 or 5, wherein, in the irradiation step, a light-shielding body forms bright and dark portions on the two-dimensional image captured at discrete times in the imaging step The light-shielding body is located between the light source and the inspection object, and blocks part of the light irradiated from the light source to the inspection object. The parts are relatively conveyed along the aforementioned conveying direction intersecting the boundary line of the aforementioned bright part and the aforementioned dark part. In the aforementioned defect type identification process, based on The data obtained by accumulating the results of machine learning related to the recognition of the type of defect contained in the segmented image is used to identify the type of defect of the inspection object. The aforementioned column segmented image means: by making the aforementioned two-dimensional image The column segmentation image formed by arranging the aforementioned columns of the positions of the aforementioned bright parts according to the order of time series; Column split image.

Images