TWI822968B - Color filter inspection device, inspection device, color filter inspection method, and inspection method - Google Patents

Abstract

The color filter inspection device includes: a defect inspection unit that detects defect candidates based on a captured image of the color filter; and a defect output unit that compares at least one physical quantity related to the detected defect candidate with a threshold value of the physical quantity. To determine whether the defect candidate is not a defect, and output the defect candidates other than the defect candidates determined to be non-defects to the neural network for defect classification.

Description

Color filter inspection device, inspection device, color filter inspection method, and inspection method

Field of invention

The present invention relates to a color filter inspection device, an inspection device, a color filter inspection method, and an inspection method.

Background of the invention

Conventionally, color filter unevenness inspection devices have been used in color filter production lines to detect the opening area and film thickness of the colored layer, which are defects that occur on color filter substrates due to changes in manufacturing conditions. Etc. area of inhomogeneity (unevenness). The color filter unevenness inspection device is a device that detects unevenness by performing image processing on a captured image of the color filter substrate. The type of unevenness (attribute items determined by shape, area, shade, etc.) is related to the cause step. When the device detects unevenness, the operator can identify the type of unevenness from the uneven image and make adjustments. The manufacturing conditions of the cause steps are used to suppress the occurrence of unevenness.

However, in order to distinguish the type of unevenness from uneven images, the operator must be skilled, and it is very difficult to make the judgment appropriately even if the operator is changed, so automation is desired. Furthermore, JP2017-54239A discloses an image classification device that can be used for appearance inspection of semiconductor substrates and the like. This technology extracts feature quantities from captured images to classify defects, and uses teacher data to classify defects through neural networks, decision trees, discriminant analysis, etc.

However, in the technology disclosed in JP2017-54239A, defects are detected based on the difference image between the captured image and the reference image. In many cases, color filter unevenness shows very subtle differences in shades. Therefore, the method using differential images has a high possibility of false detection and is not suitable for detecting color filter unevenness. In addition, in the technology disclosed in JP2017-54239A, although the characteristic amount of the defect is input to the classifier to determine the type of the defect, when constructing the classifier, it is necessary to decide which characteristic amount to use, so it is difficult to use it. .

Summary of the invention

An object of the present invention is to provide a color filter inspection device, an inspection device, a color filter inspection method, and an inspection method that can easily and accurately detect defects caused by uneven color filters and classify the defects.

means to solve problems The present invention solves the aforementioned problems through the following solutions. In addition, for ease of understanding, the description is given with symbols corresponding to the embodiments of the present invention, but the invention is not limited thereto.

The present invention is a color filter inspection device, including: a defect detection unit that detects defect candidates based on a photographed image of a color filter; and a defect output unit that combines at least one physical quantity related to the detected defect candidates and The threshold values of the aforementioned physical quantities are compared to determine whether the aforementioned defect candidates are not defects, and defect candidates other than the defect candidates determined to be non-defects are output to the neural network for defect classification.

The present invention is a color filter inspection device, including: a defect detection unit that detects defect candidates based on a photographed image of a color filter; and a defect output unit that outputs the detected defect candidates to a neural network for defect detection. Classification; and a classification determination unit that determines the result of the defect classification output from the neural network based on the first analysis result of the captured image of the color filter.

The aforementioned first analysis result may also include occurrence location information showing the occurrence location of defects of each category.

The color filter inspection device may further include a defect determination unit that determines whether the defect candidate identified as a result of the defect classification is a defect based on the second analysis result of the captured image of the color filter.

The aforementioned second analysis result may also include generation history information showing the generation history of defects of each category.

The aforementioned neural network may also include: a convolution layer that generates the first feature map from the aforementioned captured image through convolution processing; and a pooling layer that performs pooling processing to reduce the size or change of the first feature map. the second feature map; and the output layer, which outputs the results of the aforementioned defect classification.

The color filter inspection device may further include an image cropping unit that crops out a range including the defect candidates detected by the defect detection unit from the captured image.

The defect detection unit may also determine the range containing defect candidates from the following data: data obtained by differentially dividing the captured image from a plurality of directions.

The present invention is an inspection device including: a defect detection unit that detects defect candidates based on a photographed image of an object; and a defect output unit that outputs at least one physical quantity related to the detected defect candidate and a threshold value of the physical quantity Comparison is performed to determine whether the defect candidate is not a defect, and defect candidates other than the defect candidates determined to be non-defects are output to the neural network for defect classification.

The present invention is an inspection device, including: a defect detection unit that detects defect candidates based on a photographed image of an object; a defect output unit that outputs the detected defect candidates to a neural network for defect classification; and classification The determining unit determines the result of the defect classification output from the neural network based on the first analysis result of the captured image of the object.

The present invention is a color filter inspection method, which includes the following steps: a defect detection unit detects a defect candidate based on a photographed image of a color filter; and a defect output unit combines at least one physical quantity related to the detected defect candidate and The threshold values of the aforementioned physical quantities are compared to determine whether the aforementioned defect candidates are not defects, and defect candidates other than the defect candidates determined to be non-defects are output to the neural network for defect classification.

The present invention is a color filter inspection method, which has the following steps: the defect detection part detects defect candidates based on the photographed image of the color filter; the defect output part outputs the aforementioned detected defect candidates to a neural network for defect detection Classification; and a classification determination unit determines the result of the defect classification output from the neural network based on the first analysis result of the captured image of the color filter.

The present invention may further include the following steps: a step of photographing the color filter by an imaging unit; a convolution step of generating a first feature map from the captured image by the neural network through convolution processing; and performing pooling processing. The pooling step of generating a second feature map by reducing the size or variation of the first feature map.

The present invention is an inspection method, which includes the following steps: a defect detection unit detects a defect candidate based on a photographed image of an object; and a defect output unit determines at least one physical quantity related to the detected defect candidate and a threshold value of the physical quantity Comparison is performed to determine whether the defect candidate is not a defect, and defect candidates other than the defect candidates determined to be non-defects are output to the neural network for defect classification.

The present invention is an inspection method that includes the following steps: a defect detection unit detects defect candidates based on a photographed image of an object; a defect output unit outputs the detected defect candidates to a neural network for defect classification; and classification The determining unit determines the result of the defect classification output from the neural network based on the first analysis result of the captured image of the object.

According to the present invention, it is possible to provide a color filter inspection device, an inspection device, a color filter inspection method, and an inspection method that can easily and accurately detect defects in a color filter and classify the defects.

Form used to implement the invention Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings and the like. In the drawings referred to in each embodiment, the same parts or parts having the same functions are assigned the same symbols or similar symbols, and repeated descriptions thereof are omitted.

(First Embodiment) FIG. 1 is a diagram showing an embodiment of the color filter inspection device 1 of the present invention. The color filter inspection device 1 is a device used to inspect manufacturing defects during the manufacturing process of color filters. Although the example shown here is the unevenness of the color filter, which is an example of the inspection target, it can also be used to inspect the unevenness of the color filter. Defects other than unevenness. The color filter inspection device 1 of this embodiment includes an imaging unit 11, a defect detection unit 12, an image cropping unit 13, an input unit 14, a defect classification unit 15, and a learning model construction unit 17. Furthermore, the defect detection unit 12, the image cropping unit 13, the input unit 14, the defect classification unit 15 and the learning model construction unit 17 are configured by integrating dedicated programs into a computer, and the programs realize the functions of each component.

The imaging unit 11 is equipped with illumination, a camera, a transport device, etc. not shown in the figure, and photographs the color filter to be inspected, and obtains a photographed image. The photographed image captured by the imaging unit 11 is sent to the defect detection unit 12 and the image cropping unit 13 . Furthermore, the camera of the imaging unit may be a line sensor camera or an area sensor camera. The imaging unit 11 may be configured to receive reflected light from the color filter, or may be configured to receive transmitted light.

The defect detection unit 12 uses image processing to detect defect candidates that may be uneven from the image. The detection method performed by the defect detection unit 12 of this embodiment uses the method disclosed in Japanese Patent No. 4363953. That is, the defect detection unit 12 performs primary differential processing on the captured image using a spatial filter, identifies a region where unevenness is expected to exist, and uses the spatial change ratio of the grayscale value of the region as a parameter indicating the degree of unevenness. Evaluation values to detect areas where unevenness may exist.

The image cropping unit 13 crops out a predetermined range including the defect candidate detected by the defect detection unit 12 from the captured image, and obtains an uneven peripheral image. The image cropping unit 13 sends the cropped uneven peripheral image and the captured image to the input unit 14 .

The input unit 14 inputs the uneven peripheral image and the captured image to the input layer 151 of the defect classification unit 15 . By using the defect detection unit 12 to detect defect candidates and using the image cropping unit 13 to crop the image, it is possible to reduce the processing performed by the defect classification unit 15 to be described later.

The defect classification unit 15 is composed of a neural network, and uses artificial intelligence to analyze the uneven surrounding image, classify the types of unevenness, and output the defect classification. The defect classification section 15 has a learning model 16, and by comparing the uneven surrounding image with the learning model 16, the unevenness is classified (defect classification). Here, the specific form of the unevenness of the color filter, such as shape, size, area, etc., will vary depending on the cause of the unevenness. Therefore, by classifying unevenness, the cause of occurrence of unevenness (defects) can be specified. In addition, since the detection by the defect detection unit 12 may include erroneous detection, the erroneous detection may also be classified.

Furthermore, the defect classification unit 15 is provided with a learning model 16 . The learning model 16 is constructed by the learning model construction unit 17 learning the learning data 18 (each type of uneven image group) in advance.

The learning model construction unit 17 constructs a learning model from the learning materials 18 . As described above, before executing the color filter inspection, the learning model construction unit 17 studies the learning data 18 (image groups of each type of unevenness) in advance to build the learning model 19, and transfers it to the learning model 16 of the defect classification unit 15 . The learning model 19 is composed of a parameter set and the like to be used in a neural network.

Next, the operation of the color filter inspection device 1 and the structure and operation of the defect classification unit 15 will be described in more detail. FIG. 2 is a flowchart showing the flow of operations of the color filter inspection device 1 . In step (hereinafter referred to as S) 11, the imaging unit 11 photographs the color filter to obtain an image. In S12, the defect detection unit detects defect candidates. In this embodiment, the defect detection unit 12 performs primary differential processing on the image acquired in step 11 using a spatial filter, and extracts a region where unevenness is expected to exist. In S13, the image cropping unit 13 crops out a predetermined range including the defect candidate from the captured image, and obtains an uneven surrounding image. In S14 , the input unit 14 inputs the uneven peripheral image (cropped image) to the input layer of the defect classification unit 15 .

In S15, the defect classification unit 15 classifies defects from the uneven peripheral image (cropped image).

Describe the detection method of defect candidates in S12. A differential image is created by determining the differential direction on the captured image and calculating the first differential with respect to the differential direction for the grayscale value of each pixel that constitutes the image. FIG. 5 is a diagram showing an example of converting a differential image into a binary image. A predetermined threshold is set, and for the differential image, a first pixel value is assigned to the pixels with a pixel value greater than or equal to the threshold, and a second pixel value is assigned to the pixels with a pixel value less than the threshold. The differential image is made into a binary image as shown in Figure 5. FIG. 6 is a diagram showing an example of extracting a region composed of a set of adjacent pixels having a first pixel value for the binary image in FIG. 5 . Regarding this binary image, as shown in FIG. 6 , a region composed of a set of adjacent pixels having a first pixel value is extracted. The pixel group constituting the area is regarded as a set of multiple one-dimensional pixel arrays along the differential direction, and the difference between the grayscale values of the pixels located at both ends of the one-dimensional pixel array is divided by the one-dimensional The value of the length of the pixel array is used as an evaluation value indicating the non-uniformity of the one-dimensional pixel array, and a representative value among a plurality of evaluation values obtained for a plurality of one-dimensional pixel arrays is used as an evaluation value. An evaluation value indicating non-uniformity in the target area. The higher the evaluation value, the more obvious the unevenness will be seen from the surrounding area. The evaluation value for each differential direction is obtained, and if certain predetermined conditions are met, it is evaluated that there is unevenness in the evaluation target area.

The process of S12 can accurately evaluate the presence or absence of unevenness even when the size or shape of the uneven area cannot be predicted, so it is possible to hand over the defective candidate area to the class of the defect classification unit 15 with high accuracy. neural network.

A more specific structure of the defect classification unit 15 will be described together with the operation of S15 of the defect classification unit 15 . FIG. 3 is a diagram showing the structure of the neural network of the defect classification unit 15 and the learning process. The neural network of the defect classification unit 15 of this embodiment includes an input layer 151, a convolution layer 152, a pooling layer 153, a fully connected layer 154, and an output layer 155.

The input layer 151 is a layer that receives an input of the uneven peripheral image from the input unit 14 . In the example of FIG. 3 , it is shown that input of an image of 64×64 pixels can be received, but this can be changed appropriately.

The convolution layer 152 uses a coefficient matrix of any size to perform convolution processing. Here, 3×3 matrix coefficients are used for convolution processing, and correction is performed based on the bias value. In the convolution process, the first small image of 3×3 pixels is extracted from the 64×64 pixel input image, and the small image and the 3×3 coefficient matrix are used for convolution calculation, plus the deviation value The value multiplied by the deviation coefficient is applied to ReLU (Rectified Linear Unit, linear rectification function) to generate the first feature map. FIG. 3 illustrates an example of 64 convolution images created using 64 types of matrix coefficients, but this can be changed appropriately.

In the pooling layer 153, the first feature map is pooled to obtain a second feature map. The pooling process is to reduce the size or variation of the first feature map generated in the convolution layer 152 to generate a second feature map. For example, an image of 2×2 pixels is captured from the first feature map, and the maximum brightness or average brightness of the image is calculated. Specifically, it is to perform average pooling, maximum pooling, Lp pooling, etc. Figure 4 is a diagram illustrating average pooling. In the average pooling example shown in Figure 4, the size is reduced by dividing the pooled area into 2×2 pixels and averaging the brightness values.

The fully connected layer 154 in Figure 3 combines 64 second feature maps to create fully connected data.

The output layer 155 applies the parameter set (weight parameter, deviation parameter) to the fully connected data, and uses the activation function to output the classification results of nine types of defects. The parameter set (weight parameter, bias parameter) can be applied to all 64 second feature maps.

The parameter set (weight parameter, bias parameter) is set by using the backpropagation method through the learning data 18 . Calculate the output error from the learning data 18, use the least squares method for the error function and update the parameter set (weight parameter, deviation parameter) through the gradient descent method, etc., and determine the parameter set (weight parameter) by repeating learning multiple times , deviation parameter) value.

In Figure 3, the network is composed of one layer each of the convolution layer and the pooling layer. However, it is also possible to form a network composed of multiple layers of convolution layers and pooling layers, and repeat the convolution a predetermined number of times. accumulation processing and pooling processing. In addition, although the coefficient matrix used in the convolution layer is learned in advance and its value is determined and fixed, the parameter set (weight parameter, bias parameter) of the convolution layer and the fully connected data can also be learned at the same time.

In this embodiment, deep learning technology called Convolution Neural Network (CNN) can be used, and activation functions such as S-shaped functions and ReLu functions can be formed in complex layers. A network composed of forms. In addition, the above-mentioned structural example of a neural network is only an example and can be changed appropriately.

As described above, the color filter inspection device 1 of this embodiment can perform defect classification with high accuracy by using the defect classification unit 15 using a neural network. Therefore, defects in the manufacturing steps can be discovered at an early stage and improvements in the manufacturing steps can be efficiently performed.

For example, if a skilled operator sees a specific unevenness caused by foreign matter in the step of applying a photosensitive material to a substrate, the specific unevenness can be recognized, but even if it is a specific unevenness, the shape , area, shade, etc. are also very different. Therefore, although it is difficult to determine a rule for identifying unevenness based on shape, area, and shade, specific unevenness can be accurately identified by using the defect classifier 15 using a neural network. Since it is known that foreign matter in the step of coating the photosensitive material on the substrate is related to specific unevenness, an operator who knows the occurrence of specific unevenness can do this by performing the step of coating the photosensitive material on the substrate. Confirmation or removal of foreign matter and expect early improvement.

(Second Embodiment) FIG. 7 is a diagram showing a second embodiment of the color filter inspection device 1 of the present invention. The color filter inspection apparatus 1 of the second embodiment includes, in addition to the configuration of the first embodiment, a defect discrimination unit 21, a classification determination unit 22, and a defect determination unit 23 as an example of a defect output unit. These defect discrimination unit 21, classification determination unit 22 and defect determination unit 23 are configured by incorporating a dedicated program into a computer, and the program realizes the functions of each component.

The defect detection unit 12 detects defect candidates that may be uneven based on the captured image of the color filter captured by the imaging unit 11 . Although the defect detection unit 12 can also directly detect defect candidates from the captured images, it is more ideal to detect defect candidates from images that have been pre-processed to improve the determination of defects in the captured images. Precision handling.

The defect discrimination unit 21 calculates at least one physical quantity for the defect candidate detected by the defect detection unit 12 and compares the calculated physical quantity with a threshold value of the physical quantity to determine whether the defect candidate is not a defect. The defect discrimination unit 21 outputs the defect candidates other than the defect candidates determined to be non-defects to the defect classifier 15 which is a neural network to perform defect classification. On the other hand, the defect resolution unit 21 does not output the defect candidates determined to be non-defects to the defect classifier 15 .

The classification determination unit 22 determines the defect classification result output by the defect classifier 15 based on the first analysis result of the captured image of the color filter obtained in advance. Specifically, the classification determination unit 22 determines the defect classification by comparing the defect classification with the occurrence position information showing the occurrence position of defects for each classification, which is an example of the first analysis result.

The defect determination unit 23 determines whether the defect candidate for which the defect classification has been determined is a defect based on the second analysis result of the captured image of the color filter obtained in advance. Specifically, the defect determination unit 23 compares the defect candidates for which the defect classification has been determined with the occurrence history information showing the occurrence history of defects for each classification as an example of the second analysis result, thereby determining the defect candidates for which the defect classification has been determined. Whether it is a defect.

Next, the operation of the color filter inspection device 1 of the second embodiment will be described in more detail. FIG. 8 is a flowchart showing the operation flow of the color filter inspection device 1 according to the second embodiment.

First, in S21, the imaging unit 11 acquires a photographed image of the color filter.

After acquiring the captured image, in S22, the defect detection unit 12 performs pre-processing on the captured image to improve the accuracy of defect determination. Pre-processing consists of a series of processes such as shading processing, smoothing processing, radiation processing, and black and white inversion processing. Shading processing is a process of removing density unevenness from an image with density unevenness. Smoothing processing is a process that blurs the captured image to remove noise other than defect candidates contained in the captured image. The smoothing process may use, for example, a Gaussian filter. The radiation processing is a process that is necessary to perform a convolution operation when detecting a defect candidate described later, and is a process of copying the brightness value of a pixel located at an end of a captured image to the outside thereof. When convolution is performed using, for example, 3×3 pixels in which the pixel located at the end of the captured image is used as the target pixel, that is, the center pixel, there is a shortage of pixels around the target pixel. Therefore, in order to make up for this shortage, And carry out radiation treatment. The black-and-white inversion processing is a process in which the captured image is inverted in black and white when the object is determined to be a black defect, and when the object is determined to be a white defect, the captured image is not inverted in black and white. Whether the target of determination is a black defect or a white defect differs depending on the type of defect to be determined. According to the black-and-white inversion process, a white image can be detected and judged as a defect candidate regardless of the type of defect, thereby simplifying defect judgment. Furthermore, the above-mentioned shading processing, smoothing processing, radiation processing and black and white inversion processing that constitute the pre-processing can also be appropriately replaced in the order of these processes.

After performing the preprocessing, in S23, the defect detection unit 12 performs defect candidate detection processing for detecting defect candidates based on the captured image after the preprocessing. The defect candidate detection process is a series of processes such as differential filtering, binary processing, horizontal closing processing, vertical closing processing, horizontal opening processing, vertical opening processing, and labeling processing such as convolution operations. constituted. The difference filtering process of the convolution operation is a process of finding the magnitude of the change in the brightness value of each point of the captured image after pre-processing, that is, each pixel. Where there are defect candidates such as unevenness or foreign matter, the brightness value changes greatly compared with the surroundings. Therefore, the defect candidate can be detected by finding the magnitude of the change in the brightness value using differential filtering. Specifically, in the differential filtering process, each pixel of the pre-processed captured image is sequentially set as a target pixel, and each pixel of, for example, 3×3 pixels centered on the target pixel is used. The convolution operation is performed with the brightness value and a coefficient matrix of, for example, 3×3 as a filter. Through the convolution operation, the difference in brightness value between each target pixel and its adjacent pixels is calculated. Where the change in brightness value is greater, the difference value will also become larger, and the place with a larger difference value can be detected as a defect candidate.

Binary processing is based on the captured image processed by the difference filter of the convolution operation to generate a binary image in which pixels with a difference value greater than or equal to the threshold are white, and pixels less than the threshold are black. handle. Through binary processing, areas with large brightness value changes are detected as white images, that is, defect candidates, and areas with small brightness value changes, that is, other than defect candidates, are detected as black images.

The horizontal direction closing process is a process in which the white area of the binary image expands and then shrinks in the horizontal direction, that is, in the transverse direction.

The vertical direction closing process is a process in which the white area of the binary image expands and then shrinks in the vertical direction, that is, in the longitudinal direction.

The horizontal direction closing process and the vertical direction closing process can fill holes existing in the white area or connect them to nearby white areas, thereby removing noise and improving the accuracy of defect candidates.

The horizontal opening process is a process that shrinks and then expands the white area of the binary image in the horizontal direction.

The vertical opening process is a process that shrinks and then expands the white area of the binary image in the vertical direction.

According to the horizontal opening process and the vertical opening process, small white blocks can be removed, or two white areas existing nearby can be connected, so noise can be removed and the accuracy of defect candidates can be improved.

Labeling is the process of attaching a number to each pixel in the white area of a binary image. In the marking process, for example, a plurality of pixels belonging to a continuous white area are assigned the same number to each other, and the numbers assigned to the pixels in the white areas of different blocks are different from each other. By performing the marking process, defects can be classified by blob analysis in the defect resolution process described later. Furthermore, the above-mentioned differential filtering processing, binary processing, horizontal direction closing processing, vertical direction closing processing, horizontal direction opening processing, vertical direction opening processing and marking processing that constitute the defect candidate detection process can also be replaced appropriately. sequence.

After performing the defect candidate detection process, in S24, the defect resolution unit 21 individually performs defect resolution processing on the defect candidates with different numbers after the marking process, that is, for each white area. The defect resolution process is as follows: calculating at least one physical quantity for the detected defect candidate, comparing the calculated physical quantity with a threshold value of the physical quantity to determine whether the defect candidate is not a defect, and selecting defect candidates other than the defect candidates determined to be non-defects. Output to the defect classifier 15 for processing. For example, the defect resolution processing is performed by horizontal width calculation processing, vertical width calculation processing, area calculation processing, brightness difference peak calculation processing, brightness difference average calculation processing, slope calculation processing, area ratio calculation processing, streak rate calculation processing, It consists of a series of processes including roundness calculation processing, threshold value determination processing, and output processing.

The width calculation process is a process of calculating the width of the defect candidate as one of the physical quantities of the defect candidate. For example, when two points where the straight line in the X direction passing through the centroid point of the defect candidate intersects both ends of the defect candidate in the X direction have been defined, the distance in the X direction between the two points can be calculated as the horizontal width.

The aspect calculation process is a process of calculating the aspect width of the defect candidate as one of the physical quantities of the defect candidate. For example, when two points where the Y-direction straight line passing through the gravity center point of the defect candidate intersects both ends of the defect candidate in the Y-direction have been defined, the distance in the Y-direction between the two points can be calculated as the vertical width.

The area calculation process is a process of calculating the area of the defect candidate as one of the physical quantities of the defect candidate.

The brightness difference peak calculation process is a process that calculates the maximum value of the difference values among the defect candidates through the difference filter process as one of the physical quantities of the defect candidates.

The brightness difference average calculation process is a process of calculating the average value of the brightness differences of the defect candidates as one of the physical quantities of the defect candidates.

The slope calculation process is a process of calculating the change amount of the brightness difference of the defect candidate as one of the physical quantities of the defect candidate.

The area ratio calculation process is a process of calculating the area ratio of the white area and the black area of the defect candidate within a rectangular area as one of the physical quantities of the defect candidate. The aforementioned rectangular area is defined as: the whole including and circumscribed one defect candidate. At the outermost ends of the defect candidate in the X and Y directions.

The streak ratio calculation process is a process in which the defect candidate is regarded as one rectangular streak and the ratio of the long side to the short side of the streak is calculated as one of the physical quantities of the defect candidate. More specifically, the streak rate calculation process is a process of calculating the ratio of the long side to the short side of a rectangular area defined as a whole including one defect candidate and circumscribed in the X direction sum of the defect candidate. The outermost end in the Y direction.

The roundness calculation process is a process of calculating the roundness of the defect candidate as one of the physical quantities of the defect candidate. When S is the area of the defect candidate and L is the circumference of the defect candidate, the roundness can be calculated from 4πS/L ² . Furthermore, the above-mentioned horizontal width calculation processing, vertical width calculation processing, area calculation processing, brightness difference peak calculation processing, brightness difference average calculation processing, slope calculation processing, area ratio calculation processing, streak rate calculation processing and roundness calculation Processing, and the order of these processes can also be appropriately replaced.

The threshold judgment process is a process in which, for each defect candidate of a different block, each physical quantity of the defect candidate calculated through the above various calculation processes is compared with the judgment threshold value of each physical quantity preset for each physical quantity. Compare to determine whether the defect candidate is not a defect. According to the threshold determination process, defect candidates of different blocks can be divided into defect candidates determined to be non-defects and defect candidates other than defect candidates determined to be non-defects. It has not yet been determined whether defect candidates other than defect candidates judged to be non-defects are defects at this point in time. Therefore, defect candidates other than defect candidates that are determined to be non-defects may include not only defect candidates that are ultimately determined to be defects, but also defect candidates that are suspected to be non-defects.

The output processing is processing that outputs defect candidates other than defect candidates determined to be non-defects by the threshold determination process to the defect classifier 15 and does not output defect candidates determined to be non-defects by the threshold determination process to the defect classifier. 15. With this, only defect candidates other than defect candidates judged to be non-defects will become the targets of defect classification using neural networks, and defect candidates judged to be non-defects will be classified from defects using neural networks. objects are excluded.

After performing the defect resolution process, in S25, the defect classifier 15 will perform defect classification processing. The defect classification process is a process in which defect candidates other than defect candidates determined to be non-defects by the threshold value determination process are classified into any type of defects using a neural network as in the first embodiment.

After performing the defect classification process, in S26 , the classification determination unit 22 performs a classification determination process that determines the defect classification of the defect candidate output from the defect classifier 15 . The classification determination unit 22 compares the defect classification with the occurrence position information showing the occurrence position of the defect of each classification, and when the position of the classified defect candidate matches the occurrence position indicated by the occurrence position information of the corresponding classification , then determine the defect classification. On the other hand, the classification determination unit 22 determines the defect classification when the position of the classified defect candidate is inconsistent with the occurrence position indicated by the occurrence position information of the corresponding classification.

Furthermore, in addition to generating position information, the classification determination unit 22 may also perform the classification determination process using a reliability threshold of the defect classification that is individually set in advance for each classification.

After performing the classification determination process, in S27, the defect determination unit 23 performs a defect determination process that determines whether the defect candidate for which the defect classification has been determined is a defect. The defect determination unit 23 determines whether the defect candidate is a defect by comparing the defect candidate for which the defect classification has been determined with the occurrence history information showing the occurrence history of defects for each classification. The defect determination unit 23 determines that the defect candidate is a defect when the defect candidate for which the defect classification has been determined matches the production history shown in the production history information of the corresponding classification. The generation history may also be, for example, the number of consecutive generation times of defect candidates. On the other hand, when the defect candidate for which the defect classification has been determined does not match the generation history indicated by the generation history information of the corresponding classification, the defect determination unit 23 determines that the defect candidate is not a defect. Furthermore, the defect determination unit 23 determines a defect candidate whose defect classification has not been determined as a defect that does not belong to any classification.

In addition, in addition to generating history information, the defect determination unit 23 may also use defect intensity indicating the concentration of defects, that is, the brightness difference, to perform the defect determination process.

According to the second embodiment, by combining the defect classification using a neural network and the defect determination method using a criterion other than the neural network, it is possible to perform defect determination with high accuracy compared to the case where only the neural network is used. .

(Transformed form) It is not limited to the embodiment described above. Various modifications and changes are possible, and those are also within the scope of the present invention.

(1) In the first embodiment, the case where the defect candidate is detected by the defect detection unit 12 and the image is cropped by the image cropping unit 13 is exemplified, thereby reducing the processing of the defect classification unit 15 to be described later. The invention is not limited to this, and the captured image of the color filter may be directly processed by the defect classification unit 15 . In this case, the neural network of the defect classification unit 15 outputs the defect classification from the entire image.

(2) In the first embodiment, the case where the defect detection unit 12 detects defect candidates using the method disclosed in Japanese Patent No. 4363953 has been described as an example. It is not limited to this, and it may also be configured to use conventionally known defect detection methods to detect defect candidates.

The present invention can also be applied to defect inspection of objects other than color filters. For example, the present invention can also be applied to detect defects in objects such as films, glass, silicon, metal, etc. that have an appearance of the inspection object due to coating or self-luminescence. The appearance of the inspection object is not limited to the appearance that can be detected under visible light, but can also be the appearance that can be detected under infrared light or ultraviolet light. Furthermore, the present invention can also be applied to defect inspection of medical X-ray images.

Furthermore, the embodiments and modifications may be appropriately combined and used, but detailed descriptions will be omitted. In addition, the present invention is not limited to the embodiment described above.

1: Color filter inspection device 11:Photography Department 12: Defect detection department 13: Image cropping department 14:Input part 15: Defect Classification Department (Neural Network) 16: Learning model 17: Learning model construction department 18:Study materials 19: Learning model 21: Defect identification department 22: Classification determination department 23: Defect determination department 151:Input layer 152:Convolution layer 153: Pooling layer 154: Full connection layer 155:Output layer 158:Study materials S11~S15: Steps S21~S27: Steps

FIG. 1 is a diagram showing the first embodiment of the color filter inspection device 1 of the present invention. FIG. 2 is a flowchart showing the flow of operations of the color filter inspection device 1 . FIG. 3 is a diagram showing the structure of the neural network of the defect classification unit 15 and the related learning process. Figure 4 is a diagram illustrating average pooling. FIG. 5 is a diagram showing an example of converting a differential image into a binary image. FIG. 6 is a diagram showing an example of extracting a region composed of a set of adjacent pixels having a first pixel value for the binary image in FIG. 5 . FIG. 7 is a diagram showing a second embodiment of the color filter inspection device 1 of the present invention. FIG. 8 is a flowchart showing the flow of operations of the color filter inspection device 1 according to the second embodiment.

1: Color filter inspection device

11:Photography Department

12: Defect detection department

13: Image cropping department

14:Input part

15: Defect Classification Department (Neural Network)

16: Learning model

17: Learning model construction department

18:Study materials

19: Learning model

Claims (13)

A color filter inspection device, including: a defect detection unit that detects defect candidates based on a photographed image of a color filter; and a defect output unit that performs at least one physical quantity on the detected defect candidate and a threshold value of the physical quantity Compare to determine whether the aforementioned defect candidate is not a defect, and output the defect candidates other than the defect candidates determined to be non-defects to the neural network for defect classification; the classification determination unit determines based on the first of the captured images of the color filter Analyze the result to determine the result of the defect classification output from the neural network; and the defect determination unit determines whether the defect candidate determined as a result of the defect classification is based on the second analysis result of the captured image of the color filter. For defects, the aforementioned second analysis result includes generation history information showing the generation process of defects for each classification. A color filter inspection device, including: a defect detection unit that detects defect candidates based on a photographed image of the color filter; a defect output unit that outputs the detected defect candidates to a neural network for defect classification; classification a determination unit that determines the result of the defect classification output from the neural network based on the first analysis result of the captured image of the color filter; and a defect determination unit that determines the result of the aforementioned defect classification based on the second analysis result of the captured image of the color filter. As a result, it is determined whether the defect candidate identified as a result of the defect classification is a defect, and the second analysis result includes generation history information showing the generation process of defects for each classification. The color filter inspection device of claim 1 or 2, wherein the first analysis result includes occurrence location information showing the occurrence location of each classified defect. The color filter inspection device of claim 1 or 2, wherein the neural network includes: a convolution layer that generates the first feature map from the captured image through convolution processing; and a pooling layer that performs pooling processing to Reduce the size or change of the aforementioned first feature map to generate the second feature map; and the output layer, output Results of defect classification. The color filter inspection device of claim 1 or 2 further includes an image cropping unit, and the image cropping unit crops a range including the defect candidates detected by the defect detection unit from the captured image. The color filter inspection device of claim 5, wherein the defect detection unit determines the range containing defect candidates based on the following data: data obtained by differentially dividing the captured image from a plurality of directions. An inspection device including: a defect detection unit that detects defect candidates based on a photographed image of an object; and a defect output unit that compares at least one physical quantity regarding the detected defect candidate with a threshold value of the physical quantity to determine Whether the aforementioned defect candidates are not defects, and defect candidates other than defect candidates determined to be non-defects are output to the neural network for defect classification; the classification determination unit determines based on the first analysis result of the captured image of the object The result of the aforementioned defect classification is output from the aforementioned neural network; the defect determination unit determines whether the defect candidate determined as a result of the aforementioned defect classification is a defect based on the second analysis result of the photographed image of the object. The aforementioned second analysis The results include generation history information showing the generation history of defects for each category. An inspection device is provided with: a defect detection unit that detects defect candidates based on a photographed image of an object; a defect output unit that outputs the detected defect candidates to a neural network for defect classification; and a classification determination unit that performs defect classification based on The first analysis result of the photographed image of the object determines the result of the defect classification output from the neural network; and the defect determination unit determines the defect classification based on the second analysis result of the photographed image of the object As a result, whether the identified defect candidate is a defect, The aforementioned second analysis result includes generation history information showing the generation process of defects for each classification. A color filter inspection method, including the following steps: a defect detection unit detects a defect candidate based on a photographed image of the color filter; a defect output unit performs at least one physical quantity on the detected defect candidate and a threshold value of the physical quantity Compare to determine whether the aforementioned defect candidate is not a defect, and output the defect candidates other than the defect candidates determined to be non-defects to the neural network for defect classification; the classification determination unit performs the first analysis of the captured image of the color filter based on As a result, the result of the aforementioned defect classification output from the aforementioned neural network is determined; and the defect determination unit determines whether the defect candidate determined as a result of the aforementioned defect classification is a defect based on the second analysis result of the captured image of the color filter. , the aforementioned second analysis result includes generation history information showing the generation process of defects of each classification. A color filter inspection method includes the following steps: a defect detection unit detects defect candidates based on captured images of the color filter; a defect output unit outputs the detected defect candidates to a neural network for defect classification; classification The determination unit determines the result of the defect classification output from the neural network based on the first analysis result of the photographed image of the color filter; and the defect determination unit determines the result of the aforementioned defect classification based on the second analysis result of the photographed image of the color filter, To determine whether the defect candidate identified as a result of the aforementioned defect classification is a defect, the aforementioned second analysis result includes generation history information showing the generation process of defects of each classification. The color filter inspection method of claim 9 or 10 further includes the following steps: a photographing unit photographs the color filter; and the neural network generates a first feature map from the captured image through convolution processing. The convolution step; and perform pooling processing to reduce the size or change of the aforementioned first feature map to generate the pooling of the second feature map. steps. An inspection method includes the following steps: a defect detection unit detects a defect candidate based on a photographed image of an object; and a defect output unit compares at least one physical quantity of the detected defect candidate with a threshold value of the physical quantity to determine Whether the aforementioned defect candidate is not a defect, and the defect candidates other than the defect candidates determined to be non-defects are output to the neural network for defect classification; the classification determination unit determines based on the first analysis result of the captured image of the object. The aforementioned neural network outputs the result of the aforementioned defect classification; and the defect determination unit determines whether the defect candidate identified as a result of the aforementioned defect classification is a defect based on the second analysis result of the photographed image of the object, the aforementioned second analysis result Includes generation process information showing the generation process of each category of defects. An inspection method includes the following steps: a defect detection unit detects defect candidates based on a photographed image of an object; a defect output unit outputs the detected defect candidates to a neural network for defect classification; a classification determination unit determines the defect based on the object The first analysis result of the photographed image of the object is used to determine the result of the aforementioned defect classification output from the aforementioned neural network; and the defect determination unit determines the result of the aforementioned defect classification based on the second analysis result of the photographed image of the object. Whether the determined defect candidate is a defect or not, the aforementioned second analysis result includes generation history information showing the generation process of each classified defect.

Images