TWI771010B - Defect detection method, computer device, and storage medium - Google Patents

Abstract

The present application provides a defect detection method, a computer device, and a storage medium. The defect detection method includes: training an autoencoder model using defect-free images; obtaining a reconstructed image corresponding to an image to be detected by inputting the image to be detected into the autoencoder model; calculating an image error between the image to be detected and the reconstructed image based on a preset filtering threshold; inputting the image error as an independent variable into the T distribution for calculation and obtaining a calculation result; confirming that the image to be detected is a defect-free image when the calculation result falls within a preset defect judgment criterion range; and confirming that the image to be detected is a defective image when the calculation result does not fall within the range of the preset defect judgment criterion. This application can assist in detecting whether an image has defect and improve the efficiency and accuracy of defect detection.

Description

Defect detection method, computer device and storage medium

The invention relates to the field of product testing, in particular to a defect detection method, a computer device and a storage medium.

In the actual industrial production process, the surface defects of the product will have a negative impact on the appearance, comfort and performance of the product. Therefore, it is necessary to detect the appearance defect of the product to realize the quality control of the product. The traditional manual detection method relies too much on the subjective judgment of people, and also has the disadvantages of poor immediacy and high labor intensity. Therefore, it is of great practical value to sample images of actual product objects and perform automatic defect detection based on the sampled images.

In view of the above content, it is necessary to provide a defect detection method, a computer device and a storage medium, which can assist in the detection of product defects and solve the above problems.

The defect detection method includes: training a self-encoder model using a defect-free image; inputting an image to be detected into the self-encoder model, and using the self-encoder model to obtain a reconstruction corresponding to the to-be-detected image image; based on a preset filtering threshold, calculate the image error between the to-be-detected image and the reconstructed image; input the image error as an argument to the T distribution for calculation, and obtain the calculation result; when When the calculation result falls within the range of the preset defect judgment criteria, it is determined that the to-be-detected image is a defect-free image; and when the calculation result does not fall within the range of the preset defect judgment criteria, It is determined that the to-be-detected image is a defective image.

Optionally, the using the defect-free image to train the auto-encoder model includes: inputting the defect-free image into the auto-encoder model to obtain a reconstructed image of the defect-free image; using a preset optimization The objective function trains the autoencoder model, and the optimization objective function is: |X-X'| ₁ +λ|X-X'| ₂ , where X is the defect-free image and X' is the The reconstructed image of the defect-free image, λ is the weight.

Optionally, the value range of the weight λ is 0.1-10.

Optionally, the calculating the image error between the to-be-detected image and the reconstructed image based on a preset filtering threshold includes: calculating the difference between the to-be-detected image and the reconstructed image. difference image; obtaining a binary image of the difference image based on the difference image and the filtering threshold; calculating the image error based on the difference image and the binary image.

式中，△Y _i,j 為所述差值圖像，δY _i,j 為所述二值圖像，i，j表示圖元位置。 Optionally, the calculation formula of the image error is:

In the formula, ΔY i,j is the difference image, δY _i , _j is the binary image, and i,j represents the position of the primitive.

式中，

為所述過濾閾值。 Optionally, the binary image is defined as:

In the formula,

is the filtering threshold.

其中，t表示所述圖像誤差；v為自由度；

為

函數。 Optionally, the density function formula of the T distribution is:

Among them, t represents the image error; v is the degree of freedom;

for

function.

Optionally, the preset value of the degree of freedom v is 1.

The computer-readable storage medium stores at least one instruction that, when executed by a processor, implements the defect detection method.

The computer device includes a memory and at least one processor, the memory stores There is at least one instruction that, when executed by the at least one processor, implements the defect detection inspection method.

Compared with the prior art, the defect detection method, computer device and storage medium can filter small noise errors in image processing, and use the self-encoder model to detect product defects based on the properties of T distribution, thereby improving the efficiency of product inspection. and accuracy.

3: Computer device

30: Defect Detection System

301: Get Mods

302: Execute the module

31: Storage

32: Processor

33: Camera

S1~S7: Steps

In order to more clearly illustrate the technical solutions in the embodiments of the present application or in the prior art, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only It is an embodiment of the present application. For those of ordinary skill in the art, other drawings can also be obtained according to the provided drawings without any creative effort.

FIG. 1 is a flowchart of a defect detection method according to a preferred embodiment of the present application.

FIG. 2 is a functional module diagram of a defect detection system according to a preferred embodiment of the present application.

FIG. 3 is a structural diagram of a computer device according to a preferred embodiment of the present application.

In order to more clearly understand the above objects, features and advantages of the present application, the present application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments may be combined with each other in the case of no conflict.

In the following description, many specific details are set forth to facilitate a full understanding of the present application, and the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs. Specifications herein in this application The terminology used is for the purpose of describing particular embodiments only and is not intended to limit the application.

Referring to FIG. 1 , it is a flowchart of a defect detection method according to a preferred embodiment of the present application.

In this embodiment, the defect detection method can be applied to a computer device. For a computer device that needs to perform defect detection, the function for defect detection provided by the method of the present application can be directly integrated on the computer device, or the function for defect detection provided by the method of the present application can be directly integrated on the computer device. The form of software development kit (Software Development Kit, SDK) runs on the computer device.

As shown in FIG. 1 , the defect detection method specifically includes the following steps. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.

Step S1, the computer device uses the defect-free images to train the autoencoder model.

In one embodiment, the use of defect-free images to train an autoencoder model includes (a1)-(a3):

(a1) Collect a preset number of defect-free images.

In one embodiment, the defect-free image refers to an image captured of a defect-free product. Correspondingly, an image captured of a defective product is called a defective image. In one embodiment, the preset number of defect-free images may be acquired by using an industrial camera. Due to the high defect-free rate of products in the actual production process, enough (eg, 100,000) defect-free images can be taken as training samples during the just-in-time production process.

(a2) Input the defect-free image into the autoencoder model to obtain a reconstructed image of the defect-free image.

In one embodiment, the autoencoder model may be a variational autoencoder (Variational AutoEncoder, VAE) model, and the autoencoder model includes an encoder (encoder) and a decoder (decorder).

In one embodiment, the computer device may perform image processing on each defect-free image X, for example, Principal Component Analysis (PCA) dimensionality reduction, to obtain an image vector corresponding to each defect-free image X; Use the encoder to convert the each defect-free The image vector of the image X is compressed to obtain an implicit low-dimensional vector h corresponding to each defect-free image X; the decoder is used to interpret the implicit low-dimensional vector h of the defect-free image X to generate A reconstructed image X' corresponding to the defect-free image X.

(a3) Using a preset optimization objective function |X-X'| ₁ +λ|X-X'| ₂ to train the autoencoder model, where X is the defect-free image and X' is the For the reconstructed image of the defect-free image, λ is a weight, and the value of the weight λ ranges from 0.1 to 10.

In one embodiment, the error image between the defect-free image and the corresponding reconstructed image is first calculated using the calculation formula XX'; then the sparsity of the error image is measured, that is, the calculated The L1 norm of the error image: |X-X'| ₁ ; then the smoothness of the error image is measured, that is, the L2 norm of the error image is calculated: |X-X'| ₂ . In one embodiment, the computer device may calculate the error image, the L1 norm of the error image, and the L2 norm of the error image using functions in the vision library OpenCV.

In one embodiment, in order to be compatible with the smoothness and sparsity of the reconstructed image obtained by the autoencoder model, a loss function is constructed, that is, the optimization objective function: |X-X'| ₁ +λ|X-X '| ₂ , where λ is the weight, and the value of the weight λ ranges from 0.1 to 10. By taking different values, the balance between the smoothness and sparsity of the reconstructed error image of the self-encoder is achieved, and the larger the value is The reconstructed error image of the autoencoder obtained by training is smoother, and conversely, the reconstructed error image is sparser. In one embodiment, the defect-free image X and the corresponding reconstructed image X' are substituted into the optimization objective function, and the calculated value is the reconstruction error of the defect-free image X.

In one embodiment, training the self-encoder is to make the reconstruction error of the defect-free image X smaller than a preset reconstruction error metric threshold T, namely: |X-X'| ₁ +λ| X-X'| ₂ <τ. In one embodiment, the selection of the reconstruction error metric threshold depends on the expectation of the defect detection capability of the autoencoder model, and is generally selected based on a balance between defect detection recall and accuracy. If a high accuracy rate is expected, take the maximum value of the image reconstruction error in all training samples; and if the recall rate is expected to be high, a statistic based on a measure of the error-free image reconstruction error in the training sample is used as the threshold, such as assuming that during training The reconstruction error of the defect-free image obeys the Gaussian distribution, and the 90% quantile value of the Gaussian distribution can be used as the reconstruction error metric threshold T.

In step S2, the computer device inputs the image to be detected into the auto-encoder model, and uses the auto-encoder model to obtain a reconstructed image corresponding to the image to be detected.

In one embodiment, the image to be inspected may be an image obtained by photographing the product to be inspected.

In one embodiment, the computer device performs image processing (eg, PCA dimensionality reduction) on each image to be detected Y to obtain an image vector corresponding to each image to be detected Y; The image vector of the image Y is compressed to obtain the implicit low-dimensional vector s corresponding to each image Y to be detected, and the decoder is used to interpret the implicit low-dimensional vector s of the image Y to be detected, and generate The reconstructed image Y' corresponding to the to-be-detected image Y.

Step S3, the computer device calculates the image error between the image to be detected and the reconstructed image based on a preset filtering threshold.

In one embodiment, the computer device calculates a difference image between the to-be-detected image and the reconstructed image; based on the difference image and the filtering threshold, a binary value of the difference image is obtained an image; calculating the image error based on the difference image and the binary image.

In one embodiment, the preset filtering threshold may be a preset noise filtering threshold for filtering small noise during image processing.

In one embodiment, the difference image Δ Y _i,j between the image to be detected Y and the reconstructed image Y' can be calculated by using a Python program, where i, j represent the position of the primitive, for example , and use the function cv2.absdiff in the vision library OpenCV to calculate the difference image Δ Y _i,j .

，基於所述過濾閾值

定義所述差值圖像△Y _i,j 的二值圖像：

基於所述差值圖像和所述二值圖像計算所述圖像誤差，所述圖像 誤差的計算公式為：

，其中i，j表示圖元位置。 In one embodiment, in order to filter out small reconstruction error primitives, a filtering threshold is selected based on the reconstruction error of the defect-free images in the training samples

, based on the filtering threshold

Define the binary image of the difference image ΔY _i,j :

The image error is calculated based on the difference image and the binary image, and the calculation formula of the image error is:

, where i, j represent the primitive positions.

Step S4, the computer device inputs the image error as an argument to the T distribution for calculation, and obtains a calculation result.

其中，t表示所述圖像誤差；v為自由度(例如，v=1)；

為

函數。 In one embodiment, the density function formula of the T distribution is:

where t represents the image error; v is the degree of freedom (eg, v=1);

for

function.

In step S5, the computer device determines whether the calculation result falls within the range of the preset defect judgment criterion. When it is determined that the calculation result falls within the range of the preset defect judgment criterion, step S6 is executed, and when the calculation result is determined to fall within the range of the preset defect judgment criterion When it does not fall within the range of the preset defect judgment criterion, step S7 is performed.

In one embodiment, the selection method of the range of the defect judgment criterion depends on the expectation of the defect detection capability of the autoencoder model, and is generally selected based on the balance between the recall rate and the accuracy rate of defect detection, and the T distribution can be Within the 90% quantile value of , as the range of the defect judgment criteria.

In one embodiment, the distribution similarity of the T distribution has the characteristics of being similar and more similar, and dissimilar and less similar, and the T distribution is biased towards long-tailed distribution. For example, the larger the defect of the image to be inspected, the closer to the tail end of the distribution in the T distribution.

Step S6, the computer device determines that the to-be-detected image is a defect-free image.

Step S7, the computer device determines that the image to be detected is a defective image.

The above-mentioned FIG. 1 describes the defect detection method of the present application in detail. The functional modules of the software system for implementing the defect detection method and the hardware device architecture for implementing the defect detection method are introduced below with reference to FIG. 2 and FIG. 3 .

It should be understood that the embodiments are only used for illustration, and are not subject to the scope of the patent application. limitations of this structure.

Referring to FIG. 2 , it is a module diagram of a defect detection system provided by a preferred embodiment of the present application.

In some embodiments, the defect detection system 30 runs in the computer device 3 . The defect detection system 30 may include a plurality of functional modules composed of program code segments. The program codes of each program segment in the defect detection system 30 can be stored in the memory 31 of the computer device 3 and executed by at least one processor 32 to realize the defect detection function (see description in FIG. 3 for details).

In this embodiment, the defect detection system 30 can be divided into a plurality of functional modules according to the functions performed by the defect detection system 30 . The function modules may include: an acquisition module 301 and an execution module 302 . The module referred to in this application refers to a series of computer program segments that can be executed by at least one processor and can perform fixed functions, and are stored in a memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

Specifically, the acquisition module 301 can be used to acquire the defect-free image and the image to be detected captured by the camera 33 . Execution module 302 may be used to train an autoencoder model using the defect-free images. The execution module 302 can also be used to input the image to be detected into the self-encoder model, and use the self-encoder model to obtain a reconstructed image corresponding to the image to be detected: based on a preset filtering threshold, calculate Image error between the to-be-detected image and the reconstructed image; input the image error as an argument to the T distribution for calculation, and obtain a calculation result; determine whether the calculation result falls into a preset defect The range of judgment criteria, when the calculation result falls within the range of the preset defect judgment criteria, it is determined that the to-be-detected image is a defect-free image; and when the calculation result does not fall within the preset defect When judging the range of the criterion, it is determined that the image to be detected is a defective image.

Referring to FIG. 3 , it is a schematic structural diagram of a computer device according to a preferred embodiment of the present application. In a preferred embodiment of the present application, the computer device 3 includes a storage 31 , at least one processor 32 , and a camera 33 . Those skilled in the art should understand that the structure of the computer device shown in FIG. 3 does not constitute a limitation of the embodiments of the present application, and it may be a busbar type structure or a star-shaped structure. structure, the computer device 3 may also include more or less other hardware or software than shown, or different component arrangements.

In some embodiments, the computer device 3 includes a terminal capable of automatically performing numerical calculations and/or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, dedicated integrated circuits, Programmable gate arrays, digital signal processors and embedded devices, etc.

It should be noted that the computer device 3 is only an example, and other existing or future electronic products, if applicable to the present application, should also be included within the protection scope of the present application, and are incorporated herein by reference.

In some embodiments, the storage 31 is used to store program codes and various data, such as the defect detection system 30 installed in the computer device 3 , and realize high-speed and automatic completion during the operation of the computer device 3 . Access to programs or data. The storage 31 includes a Read-Only Memory (ROM), a Programmable Read-Only Memory (PROM), and an Erasable Programmable Read-Only Memory (Erasable Programmable Read). -Only Memory, EPROM), One-time Programmable Read-Only Memory (OTPROM), Electronically-Erasable Programmable Read-Only Memory (EEPROM) , Compact Disc Read-Only Memory (CD-ROM) or other optical disk storage, magnetic disk storage, magnetic tape storage, or any other computer-readable storage medium that can be used to carry or store data.

In some embodiments, the at least one processor 32 may be composed of an integrated circuit, for example, may be composed of a single packaged integrated circuit, or may be composed of a plurality of integrated circuits packaged with the same function or different functions , including one or more central processing units (Central Processing Units, CPUs), microprocessors, digital signal processing chips, graphics processors and combinations of various control chips. The at least one processor 32 is the control core (Control Unit) of the computer device 3, and uses various interfaces and lines to connect the various components of the entire computer device 3, By running or executing the programs or modules stored in the storage 31 and calling the data stored in the storage 31, various functions of the computer device 3 and processing data are performed, such as the function of performing defect detection.

Although not shown, the computer device 3 may also include a power source (such as a battery) for supplying power to various components. Preferably, the power source may be logically connected to the at least one processor 32 through the power management device, so as to realize the realization through the power management device. Manage charging, discharging, and power management functions. The power supply may also include one or more of a DC or AC power source, a recharging device, a power failure detection circuit, a power converter or inverter, a power supply status indicator, or any other element. The computer device 3 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

It should be understood that the embodiments are only used for illustration, and are not limited by this structure in the scope of the patent application.

The above-mentioned integrated units implemented in the form of software function modules can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, and includes several instructions for causing a computer device (which may be a server, a personal computer, etc.) or a processor (processor) to execute parts of the methods described in the various embodiments of the present application.

In a further embodiment, referring to FIG. 3 , the at least one processor 32 can execute the operating system of the computer device 3 and various installed applications (such as the defect detection system 30 ), program codes, etc., For example, the above-mentioned modules.

The storage 31 stores program codes, and the at least one processor 32 can call the program codes stored in the storage 31 to execute related functions. For example, each module described in FIG. 2 is a program code stored in the storage 31 and executed by the at least one processor 32, so as to realize the function of each module to achieve defect detection. Purpose.

In one embodiment of the present application, the storage 31 stores one or more instructions (ie, at least one instruction), and the at least one instruction is executed by the at least one processor 32 to implement the defect shown in FIG. 1 . purpose of detection.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division, and other division methods may be used in actual implementation.

The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they can be located in one place or distributed to multiple networks. on the unit. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional module in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware, or can be implemented in the form of hardware plus software function modules.

It will be apparent to those skilled in the art that the present application is not limited to the details of the above-described exemplary embodiments, but that the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Accordingly, the embodiments are to be regarded in all respects as illustrative and not restrictive, and the scope of this application is defined by the appended claims rather than the foregoing description, and is therefore intended to fall within the scope of the claims. All changes within the meaning and scope of the equivalents of , are included in this application. Any reference sign in a claim should not be construed as limiting the claim to which it relates. Furthermore, it is clear that the word "comprising" does not exclude other units or, and the singular does not exclude the plural. Multiple units or means stated in the device claim may also be implemented by one unit or means through software or hardware. The terms first, second, etc. are used to denote names and do not denote any particular order.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application rather than limitations. Although the present application has been described in detail with reference to the above preferred embodiments, those of ordinary skill in the art should The technical solutions can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present application.

S1~S7: Steps

Claims (10)

A defect detection method, wherein the method comprises: training a self-encoder model using a defect-free image; inputting an image to be detected into the self-encoder model, and using the self-encoder model to obtain the image to be inspected image corresponding reconstructed image; based on a preset filtering threshold, calculate the image error between the to-be-detected image and the reconstructed image; input the image error as an argument to the T distribution for calculation, and obtain calculation result; when the calculation result falls within the range of the preset defect judgment criterion, determine that the image to be inspected is a defect-free image; and when the calculation result does not fall within the preset defect judgment criterion , the image to be detected is determined to be a defective image. The defect detection method according to claim 1, wherein the using the defect-free image to train the self-encoder model comprises: inputting the defect-free image into the self-encoder model, and obtaining the defect-free image Reconstructing an image; using a preset optimization objective function to train the autoencoder model, the optimization objective function is: |X-X'| ₁ +λ|X-X'| ₂ , where X is the Defective image, X' is the reconstructed image of the defect-free image, λ is the weight. The defect detection method according to claim 2, wherein the value range of the weight λ is 0.1-10. The defect detection method according to claim 1, wherein calculating the image error between the image to be inspected and the reconstructed image based on a preset filtering threshold comprises: calculating the image to be inspected and the difference image between the reconstructed image; based on the difference image and the filtering threshold, a binary image of the difference image is obtained; based on the difference image and the binary image The image calculates the image error. The defect detection method according to claim 4, wherein the calculation formula of the image error is:

In the formula, ΔY i,j is the difference image, δY _i , _j is the binary image, and i,j represents the position of the primitive. The defect detection method according to claim 5, wherein the binary image is defined as:

In the formula,

is the filtering threshold. The defect detection method according to claim 1, wherein the density function formula of the T distribution is:

Among them, t represents the image error; v is the degree of freedom;

for

function. The defect detection method according to claim 7, wherein the preset value of the degree of freedom v is 1. A computer-readable storage medium, wherein the computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by a processor, implements the defect detection method according to any one of claim items 1 to 8. A computer device, wherein the computer device includes a memory and at least one processor, the memory stores at least one instruction, and when the at least one instruction is executed by the at least one processor, implements as claimed in items 1 to 8 The defect detection method described in any one of the above.

Images