TWI791970B - Defect detection method and defect detection device - Google Patents

Abstract

A defect detection method includes: inputting an to-be tested image to a reconstruction model; outputting the an initial reconstruction map by the reconstruction model; inputting the initial reconstruction map to the reconstruction model; outputting a complete reconstruction map by the reconstruction model; comparing each of the plurality of pixels corresponding to the complete reconstruction map and the to-be tested image to generate a plurality of deviation values; generating an deviation image based on the deviation values; inputting the deviation image to a regression model, and outputting a defect position by the regression model.

Description

Defect detection method and defect detection device

The present invention relates to a detection method and a detection device, in particular to a defect detection method and a defect detection device.

Generally speaking, when performing defect detection, the current defect detection is mainly divided into two categories: traditional image processing methods and recognition systems based on deep learning.

When using traditional image processing methods to detect defects, if the production line needs to be replaced, the machine personnel need to continuously adjust the parameters. The parameter adjustment process requires a lot of time and experienced personnel.

The method of deep learning identification system for defect detection is to learn the best parameters from data, and does not require experienced personnel to adjust the parameters of the machine, and the accuracy is usually significantly higher than the traditional image processing method, but in the replacement During the production line, new labeling data is required for re-learning, and different defects need to contain a certain amount, which will cause the recognition rate of specific defects to be lower than others. Usually, the types of defects with poor recognition rates are defects with low data quantity types, but the probability of occurrence of all defects in the production process is inconsistent, and in some cases, defects with a small probability of occurrence may be more important.

However, in the above-mentioned flaw detection method, the error threshold (threshold) needs to be manually adjusted. If the flaws in the input image are too large, the effect of the restored complete image (golden image) will not be good, making the error threshold more difficult. Adjustment. Therefore, how to automatically select defect features and automatically adjust error thresholds has become one of the problems to be solved in this field.

In order to solve the above problems, an aspect of the present disclosure provides a defect detection method. The defect detection method includes: inputting an image to be tested into a reconstruction model, and the reconstruction model outputs an initial reconstruction image; inputting the initial reconstruction image into the reconstruction model, and the reconstruction model outputs a complete reconstruction image; comparing the complete reconstruction image with the image corresponding to the image to be tested Generate a plurality of error values for each of the plurality of pixels; generate an error image according to the error values; and input the error image to a regression model, and the regression model outputs a defect position.

Another aspect of the present invention is to provide a defect detection device, which includes a processor. The processor is used to input an image to be tested into a reconstruction model, the reconstruction model outputs an initial reconstruction image, the initial reconstruction image is input into the reconstruction model, the reconstruction model outputs a complete reconstruction image, and the complete reconstruction image is compared with the corresponding image in the test image Generate a plurality of error values for each of the plurality of pixels; generate an error image according to the error values; and input the error image to a regression model, and the regression model outputs a defect position.

To sum up, the flaw detection method and flaw detection device in this case can use the error value of the complete reconstruction map and the image to be tested to generate an error image, and input the error image into the regression model to learn the best parameters, so as to train an accurate Therefore, the defect detection method and the defect detection device can determine the defect position of the newly input image, saving the manpower of manually adjusting the threshold when the production line changes products.

The following description is a preferred implementation of the invention, and its purpose is to describe the basic spirit of the invention, but not to limit the invention. For the actual content of the invention, reference must be made to the scope of the claims that follow.

It must be understood that words such as "comprising" and "including" used in this specification are used to indicate the existence of specific technical features, values, method steps, operations, components and/or components, but do not exclude possible Add more technical characteristics, values, method steps, operation processes, components, components, or any combination of the above.

Words such as "first", "second", and "third" used in the claims are used to modify the elements in the claims, and are not used to indicate that there is an order of priority, an antecedent relationship, or an element An element preceding another element, or a chronological order in performing method steps, is only used to distinguish elements with the same name.

Please refer to FIGS. 1-2. FIG. 1 is a flow chart of a defect detection method 100 according to an embodiment of the present invention. FIG. 2 is a schematic diagram illustrating a defect detection method according to an embodiment of the present invention.

In an embodiment, the defect detection method 100 may be executed by a defect detection device, and the defect detection device includes a processor and a storage device.

In one embodiment, the processor can be used to execute the defect detection method 100 . The processor can be implemented as, for example, a microcontroller, a microprocessor, a digital signal processor, an application specific integrated circuit (ASIC), or a logic circuit.

In one embodiment, the storage device is used to store various images, and the storage device may be a read-only memory, flash memory, floppy disk, hard disk, optical disk, flash drive, magnetic tape, a database that can be accessed by the network, or Those skilled in the art can easily think of storage media with the same function.

In step 110, the processor inputs an image to be tested IMG0 to a reconstruction model MD, and the reconstruction model MD outputs an initial reconstruction image IMG1.

In an embodiment, please refer to FIGS. 3 and 4A-4C. FIG. 3 is a schematic diagram illustrating the image to be measured IMG0 according to an embodiment of the present invention. Figures 4A-4C are schematic diagrams illustrating reconstructed images according to an embodiment of the present invention.

In one embodiment, the image to be tested IMG0 is shown in FIG. 3 , and the image to be tested IMG0 is generated by a synthetic model. Synthetic models can be implemented using known neural network-like implementations. More specifically, the processor first obtains a complete image IMG5 (golden image), and synthesizes a known defect IMG6 with the complete image. The synthesis method can use a mixup algorithm or a Generative Adversarial Nets (GAN) ) etc. to generate the image to be tested IMG0. In one embodiment, the processor can read the original defect image of the known defect IMG6 (including the position, shape and size of the defect) from a defect database, and mask the known defect IMG6 to obtain the defect mask Mask image IMG4, after inputting the original known defect IMG6, defect mask image (known defect range) IMG4 and complete image IMG5 into the synthesis model CM, a synthetic defect image is generated, and this synthetic defect image can be used as the image to be tested IMG0 . In one embodiment, the composite model CM can be realized by using known software image processing software or image processing circuit.

In addition, the blemish mask image IMG4 refers to a mask, and is not limited to be presented as an image.

In one embodiment, the reconstruction model MD is realized by an autoencoder (AutoEncoder) algorithm or a variational autoencoder (Variaional autoencoder, VAE). However, the present invention is not limited thereto, as long as it is a computer vision technology that can deduce and calculate real scenes, complete two-dimensional images and/or three-dimensional spatial information of objects from part or all of the images captured by the camera, and reconstruct them can be applied.

In one embodiment, the regression model RM is implemented by a linear regression algorithm or a ridge regression algorithm. However, the present invention is not limited thereto.

In step 120, the processor inputs the initial reconstructed graph IMG1 to the reconstructed model MD, and the reconstructed model MD outputs a complete reconstructed graph IMG1c.

In one embodiment, the processor executes a recursive reconstruct algorithm after inputting the initial reconstructed image IMG1 to the reconstructed model MD to obtain a complete reconstructed image. More specifically, as shown in Figures 4A-4C, after the initial reconstruction image IMG1 is generated, the processor sends the initial reconstruction image IMG1 to the reconstruction model MD again, and the reconstruction model MD generates an intermediate reconstruction image IMG1a based on the initial reconstruction image IMG1 ; Then, repeat the reconstruction model MD to output a current intermediate reconstruction map (for example, the intermediate reconstruction map IMG1b), and then input this current intermediate reconstruction map to the operation of the reconstruction model MD (that is, the process of the frame selection part 210), until a stop is satisfied Conditions, the current intermediate reconstruction image of the last one is regarded as the complete reconstruction image IMG1c.

Taking Figures 4A~4C as an example, after the processor sends the initial reconstruction image IMG1 to the reconstruction model MD again, the reconstruction model MD generates an intermediate reconstruction image IMG1a, and then, the processor sends the intermediate reconstruction image IMG1a to the reconstruction model MD, and the reconstruction model The MD generates an intermediate reconstruction image IMG1b, and then, the processor sends the intermediate reconstruction image IMG1b to the reconstruction model MD, and the reconstruction model MD generates a complete reconstruction image IMG1c. Wherein, the stop condition may refer to a preset number of times for iterating the intermediate reconstruction image repeatedly. In this way, the reconstruction model MD can be helped to find the restoration target (such as the defective part) by repeatedly iterating the intermediate reconstruction image.

In addition, by using the cyclic reconstruction mechanism (ie, the process of the frame selection part 210), the reconstructed image (ie, the intermediate reconstruction map generated multiple times) is re-input into the reconstruction model MD for multiple restorations, which can enhance the ability to eliminate large defects.

In step 130, the processor compares each plurality of pixels corresponding to the complete reconstruction image IMG1c and the image under test IMG0 to generate a plurality of error values, and generates an error image IMG2 according to these error values.

In an embodiment, please refer to FIG. 5 , which is a schematic diagram illustrating an error image IMG2 according to an embodiment of the present invention. For example, the processor compares each plurality of pixels corresponding to the complete reconstruction image IMG1c and the image to be tested IMG0, and subtracts the grayscale value of each corresponding pixel in the complete reconstruction image IMG1c from the image to be tested IMG0 to obtain multiple The error values are used to generate the error image IMG2.

In step 140, the processor inputs the error image IMG2 to a regression model RM, and the regression model RM outputs a defect position IMG3.

In one embodiment, the regression model RM can apply a known algorithm, such as a simple linear regression model, a multiple linear regression model . . . and so on.

In an embodiment, please refer to FIG. 6 . FIG. 6 is a schematic diagram illustrating a defect position IMG3 (a range) according to an embodiment of the present invention. After the regression model RM outputs the blemish position IMG3, the processor subtracts the blemish position IMG3 from each complex number of pixels corresponding to the known blemish IMG6, and differentiates the obtained subtracted values. The detailed example is as follows: The measured image IMG0 generates the initial reconstruction image IMG1 through the reconstruction model MD (the concept here is represented by a symbolic formula: IMG1=MD(IMG0)), and the error image IMG2 (here The concept of the symbol is represented by the formula: IMG2=|IMG0-IMG1|), the error image IMG2 generates the defect position IMG3 through the regression model RM (the concept here, the formula is represented by the symbol: IMG3=RM(IMG2), compare the defects The position IMG3 corresponds to each pixel in the known defect range to generate the final error (that is, "Error", hereinafter represented by the symbol ERR). In one example, the final error ERR is determined by the defect mask image IMG4 and the defect position IMG3 It is obtained by subtraction (the concept here is represented by symbols and the formula is: ERR=IMG4-IMG3), and the final error ERR is calculated, such as differential operation, The gradient image of the defect mask image IMG4 and the gradient image of the defect position IMG3 can be obtained, and the gradient obtained by the regression model RM and the reconstruction model MD can be pushed back through the back propagation algorithm (Back Propagation), so as to update the regression model RM and reconstruction model MD to adjust regression model RM and/or reconstruction model MD.

For example, the defect mask image IMG4 and the defect position IMG3 are both a WxHx3 matrix, where the symbol W is the width of the matrix, and the symbol H is the height of the matrix. After subtracting the two matrices, the error value can be obtained, and the error value is differentiated Back Propagation can be used to adjust the parameters of the previous model, including the parameters of the reconstruction model and a threshold value of the regression model RM. This threshold value can be used to judge the difference between the defect position IMG3 and the image IMG0 to be tested. The error greater than the threshold value means that the regression model RM judges the position as a defect, and the regression model RM should be able to make the defect position IMG3 as close as possible after learning. Approximate flaw mask image (known flaw range) IMG4.

In one embodiment, when the defect position IMG3 is sufficiently similar to the known defect IMG6 (the similarity can be calculated by the Euclidean distance (L2 distance) or the structural similarity (structural similarity, SSIM) of the two images, etc. ), which means that the defect detection device has been trained, that is, the training stage is over (steps 110-140 are completed). In the detection stage, the defect detection device can accurately predict the defect position of a new input image.

In other words, the defect mask image IMG4 is known when the image to be tested is generated, and is used to train the regression model RM and/or the reconstruction model MD of the defect detection device. When the defect position IMG3 is sufficiently similar to the known defect IMG6, it means that After training the defect detection device, the training phase ends (steps 110-140 are completed). In the detection (or testing) stage, an unknown image is input into the trained regression model RM and/or reconstruction model MD to predict a predicted defect position in the unknown image.

The methods of the present invention, or specific forms or parts thereof, may exist in the form of program codes. The code may be contained in a physical medium, such as a floppy disk, compact disc, hard disk, or any other machine-readable (such as computer-readable) storage medium, or a computer program product without limitation in external form, wherein, When the program code is loaded and executed by a machine, such as a computer, the machine becomes a device for participating in the present invention. Code may also be sent via some transmission medium, such as wire or cable, optical fiber, or any type of transmission in which when the code is received, loaded, and executed by a machine, such as a computer, that machine becomes the Invented device. When implemented on a general-purpose processing unit, the code combines with the processing unit to provide a unique device that operates similarly to application-specific logic circuits.

To sum up, the flaw detection method and flaw detection device in this case can use the error value of the complete reconstruction map and the image to be tested to generate an error image, and input the error image into the regression model to learn the best parameters, so as to train an accurate Therefore, the defect detection method and the defect detection device can determine the defect position of the newly input image, saving the manpower of manually adjusting the threshold when the production line changes products.

Although the present invention has been disclosed above in terms of implementation, it is not intended to limit the present invention. Anyone skilled in this art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection of the present invention The scope shall be defined by the appended patent application scope.

100: Blemish Detection Methods

110~140: Steps

IMG0: image to be tested

MD: rebuild model

IMG1: Initial reconstruction map

IMG2: error image

RM: regression model

IMG3: blemish location

IMG1: Initial reconstruction map

IMG1a, IMG1b: Intermediate reconstruction maps

IMG1c: Complete reconstruction map

IMG2: error image

IMG3: blemish location

IMG4: Blemish Mask Image

IMG6: Known flaws

IMG5: Full Image

FIG. 1 is a flowchart illustrating a defect detection method according to an embodiment of the present invention. FIG. 2 is a schematic diagram illustrating a defect detection method according to an embodiment of the present invention. The third is a schematic diagram showing the image to be tested according to an embodiment of the present invention. 4A-4C are schematic diagrams showing reconstructed images according to an embodiment of the present invention. FIG. 5 is a schematic diagram illustrating an error image according to an embodiment of the present invention. Fig. 6 is a schematic diagram illustrating the location of defects according to an embodiment of the present invention.

100: Blemish Detection Methods

110~140: Steps

Claims (5)

A defect detection method, comprising: inputting an image to be tested to a reconstruction model, and the reconstruction model outputs an initial reconstruction image; inputting the initial reconstruction image to the reconstruction model, and the reconstruction model outputs a complete reconstruction image; comparing the complete reconstruction Mapping each plurality of pixels corresponding to the image to be tested to generate a plurality of error values; generating an error image according to the error values; and inputting the error image into a regression model, and the regression model outputs a defect position; Wherein, after the step of outputting the defect position by the regression model, it further includes: comparing the defect position with a plurality of corresponding pixels in a defect mask image to generate a final error; and calculating the final error to adjust The regression model or the reconstructed model; wherein, after the step of inputting the initial reconstructed image into the reconstructed model, further comprising: repeatedly outputting the reconstructed model into an intermediate reconstructed image, and then inputting the intermediate reconstructed image into the reconstructed model The operation until a stopping condition is satisfied, the last intermediate reconstruction image is regarded as the complete reconstruction image. The defect detection method described in claim 1 of the patent application, wherein the defect mask image is known when the image to be tested is generated. The defect detection method described in claim 1 of the patent application further includes: inputting an unknown image into the trained regression model or the reconstruction model to predict a predicted defect position in the unknown image. A defect detection device, comprising: a processor for inputting an image to be tested into a reconstruction model, the reconstruction model outputs an initial reconstruction image, inputs the initial reconstruction image into the reconstruction model, and the reconstruction model outputs a complete reconstruction image, comparing the complete reconstruction image with each plurality of pixels corresponding to the image to be tested to generate a plurality of error values; generating an error image based on the error values; and inputting the error image to a regression model, the The regression model outputs a defect location; wherein, the processor is further used to compare the defect location with a plurality of corresponding pixels in a defect mask image to generate a final error; and operate on the final error to adjust the regression model or the reconstructed model; wherein, the processor is further configured to repeatedly output the reconstructed model into an intermediate reconstructed graph, and then input the intermediate reconstructed graph into the reconstructed model, until a stopping condition is met, and the last The intermediate reconstructed image is regarded as the complete reconstructed image. The defect detection device as described in item 4 of the scope of the patent application, wherein the image to be tested is known.

Images