KR20220167824A - Defect detection system and method through image completion based on artificial intelligence-based denoising - Google Patents

Abstract

The present invention relates to a defect detection system and method, and more specifically, to a defect detection system and method through image construction based on artificial intelligence-based denoising, for providing an environment in which noise is removed by using a model created by artificial intelligence based on a captured object image and defect detection is accurately determined through the noise-removed image. According to an embodiment of the present invention, the defect detection system through image construction based on artificial intelligence-based denoising comprises: a receiving unit for receiving a captured image created by capturing an object; and a defect detection determination unit for removing noise included in the captured image, removing defective elements in the noise-removed image, and then constructing an image consisting of only normal elements to determine detection of defective objects, wherein the defect detection determination unit includes an artificial intelligence-based learning model for removing noise from the captured image and constructing the image consisting of only normal elements. In addition, according to one embodiment of the present invention, the defect detection method through image construction based on artificial intelligence-based denoising comprises: a captured image reception step in which the receiving unit receives the captured image created by capturing the object; and a defect detection determination step in which the defect detection determination unit removes noise included in the captured image, removes the defective elements in the noise-removed image, and then constructs the image consisting of only normal elements to determine object defect detection.

Description

Defect detection system and method through image construction according to artificial intelligence-based noise removal

The present invention relates to a defect detection system and method, and more particularly, to remove noise using a model generated through artificial intelligence based on an image of a photographed object, and to determine defect detection through the removed image. It is about a system and method for detecting defects through image construction according to artificial intelligence-based noise removal that can provide an environment in which this is accurately achieved.

An optical system (eg, vision) refers to a system that collects image data for a specific purpose in a factory. For example, this is to detect defects, abnormalities, and the like based on images to be photographed. The optical system is composed of a camera module, a lens, and the like, and lighting is installed around the optical system to generate brightness for capturing a desired image.

However, defect detection systems using existing optical systems clearly define the shape, size, and degree of defects to be detected, such as surface flaw detection, and are suitable for inspection environments limited to the definition, and are suitable for processes such as assembly. Since there is a great diversity of assembly results in , it is not suitable in an environment where judgment is required based on a set rule.

In addition, the conventional object recognition technology focuses on the fact that the recognition performance of each algorithm varies depending on the properties of an object, and clusters various object image data and then selectively uses the most effective algorithm to improve recognition performance. Since the multiclass classification method is mainly used, there is a limit to practical and diverse utilization after object recognition.

Republic of Korea Patent Registration No. 10-0459893 Republic of Korea Patent Publication No. 10-2014-0095333

SUMMARY OF THE INVENTION The present invention is intended to solve the above problems, and to provide a detection system and method capable of providing an environment for accurately and quickly detecting defects in objects in any environment.

However, the technical problem to be achieved by the present embodiment is not limited to the technical problems described above, and other technical problems may exist.

According to an embodiment of the present invention, a defect detection system through image construction according to artificial intelligence-based noise removal includes a receiver for receiving a photographed image generated by photographing an object; and a defect detection determination unit configured to remove noise included in the captured image, remove defective elements in the noise-removed image, and construct an image composed of only normal elements to determine object defect detection. The determination unit may include an artificial intelligence-based learning model for removing noise from the captured image and constructing an image composed of only normal elements.

In addition, the defect detection and discrimination unit is characterized in that the artificial intelligence-based learning model is configured based on deep learning.

In addition, the defect detection and discrimination unit is characterized in that it additionally performs image acquisition composed of a consistent form of color based on the noise-removed image.

In addition, the defect detection and discrimination unit includes a noise elimination model among artificial intelligence-based learning models, and the denoising model learns features of the captured image and determines a part excluding the learned feature as noise to perform noise elimination. characterized by

In addition, the denoising model is characterized in that it includes an encoder configured with a structure in which a deep learning-based convolutional neural network (CNN) is continuously connected and has a final output value as all connected layers (Dense Layer).

In addition, the denoising model includes a decoder configured with a structure in which a deep learning-based data extensible convolutional neural network (Dilated CNN) is continuously connected and does not have a final output value as all connected layers (Dense Layer). to be characterized

In addition, the defect detection and discrimination unit includes a defect recognition model among artificial intelligence-based learning models, and the defect recognition model finds and removes defective elements based on the denoised image, builds an image with only normal elements, and removes only noise. It is characterized in that defect detection and determination is performed through comparison with the image.

In addition, the defect recognition model is characterized in that it includes a layer (reduction layer) in which a convolutional neural network (CNN) based on deep learning is continuously connected and the input data dimension is maintained as it is.

In addition, the defect recognition model is characterized by including a feature extract layer in which deep learning-based convolutional neural networks (CNNs) are continuously connected and only features are extracted without changing the data dimension.

In addition, the defect recognition model is characterized by including an upscaling layer in which deep learning-based data scalable convolutional neural networks (Dilated CNNs) are continuously connected and output data equal to the input size.

In addition, according to an embodiment of the present invention, a defect detection method through image construction based on artificial intelligence-based noise removal includes a captured image receiving step of receiving a captured image generated by the receiving unit photographing an object; and a defect detection and discrimination step in which the defect detection and discrimination unit removes noise included in the captured image, removes defective elements in the noise-removed image, and constructs an image composed of only normal elements to determine object defect detection. to be characterized

According to the present invention, a defect detection system through image construction based on artificial intelligence-based noise removal can provide an environment for accurately and quickly detecting and discriminating object defects in any environment.

In addition, it is possible to improve defect detection accuracy compared to the existing environment.

In addition, it is possible to provide a defect detection environment for assembly parts made of different materials compared to the existing environment.

However, the effects of the present invention are not limited to the above-mentioned effects, and effects not mentioned will be clearly understood by those skilled in the art from this specification and the accompanying drawings.

1 is a conceptual diagram of a defect detection system through image construction based on artificial intelligence-based noise removal as an embodiment of the present invention.
Figure 2 is an example of noise removal of a defect detection system through image construction according to artificial intelligence-based noise removal as an embodiment of the present invention.
3 is a diagram for explaining noise removal of a defect detection system through image construction according to artificial intelligence-based noise removal as an embodiment of the present invention.
4 is an exemplary view of a noise removal model of a defect detection system through image construction according to artificial intelligence-based noise removal as an embodiment of the present invention.
5 is an example of a defective area of a defect detection system through image construction based on artificial intelligence-based noise removal as an embodiment of the present invention.
6 is a diagram for explaining defect detection in a defect detection system through image construction based on artificial intelligence-based noise removal as an embodiment of the present invention.
7 is an exemplary view of a defect recognition model of a defect detection system through image construction based on artificial intelligence-based noise removal as an embodiment of the present invention.
8 is a flowchart illustrating a method of detecting defects through image construction based on artificial intelligence-based noise removal as an embodiment of the present invention.

The terms or words used in this specification and claims should not be construed as being limited to ordinary or dictionary meanings, and the inventors may appropriately define the concept of terms in order to explain their invention in the best way. It should be interpreted as a meaning and concept consistent with the technical idea of the present invention based on the principle that there is.

Therefore, the embodiments described in this specification and the configurations shown in the drawings are only one of the most preferred embodiments of the present invention, and do not represent all the technical ideas of the present invention, so that they can be substituted at the time of this application. It should be understood that there may be many equivalents and variations.

Hereinafter, a defect detection system and method through image construction according to artificial intelligence-based noise removal according to the present invention will be described with reference to the drawings.

1 is a conceptual diagram of a defect detection system through image construction based on artificial intelligence-based noise removal as an embodiment of the present invention.

A defect detection system through image construction based on artificial intelligence-based noise removal according to the present invention basically includes a receiver and a defect detection and determination unit.

Referring to FIG. 1, more specifically, the present invention provides a receiving unit for receiving a captured image generated by photographing an object, removing noise included in the captured image, and removing defective elements in the noise-removed image, and then normalizing the image. and a defect detection and discrimination unit for determining object defect detection by constructing an image composed of only elements, wherein the defect detection and discrimination unit removes noise from a captured image and constructs an image composed of only normal elements based on artificial intelligence. Include a learning model.

In addition, the defect detection system may further include a storage unit for storing information necessary for implementing detection and determination and a transmitter for transmitting information to an external device (eg, an object image photographing unit, a display device, etc.), wherein the receiver unit may receive information from an external device.

In addition, as an example, the object image capture means among external devices may be composed of devices for image capture, and may be a vision camera (including image sensors, lenses, shutters, and control devices) or a group consisting of a plurality of vision cameras. . However, it is not limited thereto, and can be variously modified at a level obvious to those skilled in the art.

In addition, the image received by the defect detection system may be created by capturing an object image, or may be an image created by an external server or computer program.

In addition, as an example, the display device may be a display device, a terminal device, a computing device, a smart TV, a PDA, and the like, and by displaying the display through the display, the user can easily check the standard defect detection environment, but is not limited thereto. In addition, the display device may be variously modified at a level obvious to those skilled in the art.

Figure 2 is an example of noise removal of a defect detection system through image construction according to artificial intelligence-based noise removal as an embodiment of the present invention, Figure 3 is an image according to artificial intelligence-based noise removal as an embodiment of the present invention 4 is a diagram for explaining noise removal of a defect detection system through construction, and FIG. 4 is an exemplary view of a noise removal model of a defect detection system through image construction according to artificial intelligence-based noise removal as an embodiment of the present invention.

Referring to FIGS. 2 to 4, the defect detection and discriminating unit has an artificial intelligence-based learning model based on deep learning, and the defect detection and discriminating unit has a consistent form based on a noise-removed image. Image acquisition composed of colors may be additionally performed.

In addition, the defect detection and discrimination unit includes a noise elimination model among artificial intelligence-based learning models, and the denoising model learns characteristics of the captured image and determines a portion excluding the learned characteristics as noise to perform noise elimination. may be

In addition, the denoising model includes an encoder configured with a structure in which a deep learning-based convolutional neural network (CNN) is continuously connected and has a final output value as all connected layers (Dense Layer), and deep learning-based data extension It includes a decoder configured with a structure in which this possible convolutional neural network (Dilated CNN) is continuously connected and does not have a final output value as all connected layers (Dense Layer).

In addition, as an example of the deep learning-based convolutional neural network (CNN) used in the denoising model, a mixed network structure is constructed based on ResNet and the Inception concept of GoogLeNet is added, which is an image due to down-sampling In order to avoid loss of detail information of , based on the concept of Dilated Convolution, receptive fields of various sizes may be applied to the input image, and values of each receptive field may be mixed by performing a concatenated operation.

In addition, by applying ResNet's Skip Connection, small changes (fluctuations) can be easily detected, which can be easily optimized for deep networks while applying the structure of ResNet, and accuracy can be improved due to the increased network depth.

In addition, the size of the receptive field can be expanded without loss of resolution by using the concept of Dilated Convolution. For example, only the coefficient of the position of an arbitrary rectangle for an image divided into pixels exists in the entire reactive field, and all others are 0. , and the dilated convolution has a parameter called the dilation rate, which defines the interval between kernel values for the basic convolutional layer.

Also, as an example, since down-sampling causes loss of information in convolution operation, dilated convolution may be applied instead of applying the pooling layer.

In addition, as an example, in defining a network of structures, inputs go into three different dilated convolutional layers, respectively, concatenate the result after each dilated convolution operation, and then perform a 1 x 1 convolution operation to utilize ResNet's Skip Connection to obtain an initial The input value and Add operation are performed, and the kernels of all convolutional layers of the network defined above are all defined as 3 x 3 in size, and the padding size is 1, 2, 3 in conjunction with the dilation rate of 1, 2, 3. , and since the output of the network block is the same as the input size, the output size of the entire network may also be the same. Accordingly, the output size of the dilated convolution layer may be calculated as in Equation 2 below.

로 표현되며, i는 input size, k는 커널(필터) size, p는 zero-padding, s는 stride, d는 dilation rate를 의미하고, 이에 있어서, d = p, k = 3, s = 1로 설정하고 앞서 설명한 바와 같이 input 사이즈는 output 사이즈와 동일할 수도 있다.Also, Equation 2 is

, where i is the input size, k is the kernel (filter) size, p is zero-padding, s is the stride, and d is the dilation rate, where d = p, k = 3, s = 1 As set up and described earlier, the input size may be the same as the output size.

In addition, as an example of the network structure, the network is a convolutional network configured without a pooling layer, and it is difficult to learn to use large-sized data as it is without a pooling process, but a batch normalization layer is also applied to maintain the color information of the image. It does not, and the data size and convolution layer size of the learnable settings can be experimentally and repeatedly performed to determine.

In addition, the Initial Convolution Layer is set to 64 channels, and in the case of the final convolution layer, the color scale is set to 3 channels and the gray scale is set to 1 channel, and all ReLU layers have the same positive and negative values of the Final Residual output. It is defined as a parametric ReLU (pReLU) in order to do so, and the network described as an example above may be learned to estimate noise (residual image) through Global Skip Connection.

In addition, as an example, in order to perform noise removal more efficiently using a defined network, the concept of U-Net is applied and the detail map (mask area) obtained through U-Net is used as outline extension information to match the original image and After concatenating, noise removal (denoising) can be performed by passing it to the input of the previously defined network.

In addition, U-Net can be used as a backbone network to extract detail maps, and through the encoding/decoding process of U-Net, the size of the feature is reduced by half and the number of dimensions is doubled as it passes through the Conv block in the encoding process. , and in the decoding process, as it passes through the Conv block opposite to the encoding process, the size of the feature doubles and the number of dimensions decreases by half. A 3 x 3 kernel is applied to sampling and up-sampling, and batch normalization is applied after conv operation. U-Net can also minimize feature information that can be lost in sub-sampling and up-sampling by using Symmetric Shortcut Connection. there is.

5 is an exemplary view of a defective area of a defect detection system through image construction according to artificial intelligence-based noise removal as an embodiment of the present invention, and FIG. 6 is an image according to artificial intelligence-based noise removal as an embodiment of the present invention. 7 is a view for explaining defect detection of a defect detection system through construction, and FIG. 7 is an exemplary view of a defect recognition model of a defect detection system through image construction based on artificial intelligence-based noise removal as an embodiment of the present invention.

5 to 7, the defect detection and discrimination unit includes a defect recognition model among artificial intelligence-based learning models, and the defect recognition model finds and removes defective elements based on noise-removed images, and only selects normal elements. It is also possible to perform defect detection and discrimination through comparison with an image from which only noise is removed by constructing an image with existing data.

In addition, the defect recognition model removes a certain area (eg, after converting to a 128 x 128 pixel size image, each of 3 x 3 areas that are not identical) from the entire image based on the denoised image Defect detection and discrimination can also be performed by constructing an image with only normal elements and comparing it with an image from which only noise has been removed.

In addition, the faulty cognitive model includes a layer (reduction layer) in which deep learning-based convolutional neural networks (CNNs) are continuously connected and input data dimensions are maintained as they are, and deep learning-based convolutional neural networks (CNNs) It includes a feature extract layer that is continuously connected and extracts only features without changing the data dimension, and a convolutional neural network (Dilated CNN) that can expand data based on deep learning is continuously connected and has the same size as the input. It includes a layer that outputs data (Upscaling Layer).

In addition, as an example, the poor recognition model is trained for image construction (completion) through clear normal images from which noise has been removed (denoising), and an image completion network (deep learning-based convolutional neural network (CNN) is continuously ), and when the inspection target (object) image comes in, it is converted into a 128 x 128 pixel size image, then the patch is removed from each area in the 3 x 3 area 9 times and the process of restoring it is performed You may.

In addition, the difference between each patch area and the restored image corresponding area is calculated, the difference is calculated as an abnormality score, the area with the large difference is determined to contain defects, and the matching difference in pixel units is calculated to obtain the pixel with the large difference. By extracting the regions and masking them on the original image, the corresponding region may be marked as a defective region.

In addition, as an example of the network structure for image construction (completion), the 32 x 32 central area of the input image patch (128 x 128) is removed (set to 0) before entering the input of DCNN (Deep Convolutional Neural Network), and the image After reconstructing the image in the network for completion (construction), the generated patch area is subtracted to obtain the difference from the corresponding area of the original image, and the absolute value of the image difference can be used as an anomaly score. there is.

In addition, the network for image completion may be composed of 17 layers as shown below, Conv(5, 1, 1, 32) - Conv(3, 1, 1, 64) - Conv(3, 1, 1, 64 ) - Conv(3, 1, 2, 128) - Conv(3, 1, 1, 128) - Conv(3, 1, 1, 128) - Conv(3, 2, 1, 128) - Conv(3, 4, 1, 128) - Conv(3, 8, 1, 128) - Conv(3, 16, 1, 128) - Conv(3, 1, 1, 128) - Conv(3, 1, 1, 128) - Bilinear Upscaling(2x) - Conv(3, 1, 1, 64) - Conv(3, 1, 1, 64) - Conv(3, 1, 1, 32) - Conv(3, 1, 1, 16) - Conv (3, 1, 1, 1) - Clip (-1, 1), where Conv (k, d, s, c) represents a convolutional layer with a kernel size of k x k, d is the dilation rate, s may mean a stride and c may mean an output channel.

In addition, as an example of the network structure, the network as a whole consists of 17 layers, the resolution of the feature map after the third layer is reduced by half by stride convolution, and layer 7 to increase the receptive field of the output neuron. In layer 10, dilated convolution can be continuously applied, and in layer 13, bilinear rescaling is performed through convolution operation for upscaling to the input size, mirror padding is used for all convolutional layers, and activation function As an ELU (Exponential Linear Unit) activation function may be applied.

In addition, the network is trained based on the L1 reconstruction loss, and the 32 x 32 central region defined by the binary mask M is weighted differently from the rest of the region, and when X is the image patch to be inspected, the network is It may also be trained through Equation 1.

로 표현되며, 여기서

는 element-wise 행렬곱을 의미하며,

은 M의 complement mask를 의미하고,

는 중심 영역과 나머지 영역 사이의 가중치 파라미터이며, N은 X의 픽셀 수,

는 L1 행렬정규화(matrix normalization)을 의미한다.In addition, Equation 1 shows a loss function,

is expressed as, where

means element-wise matrix multiplication,

Means the complement mask of M,

is the weighting parameter between the center area and the rest area, N is the number of pixels in X,

Means L1 matrix normalization.

In addition, the image completion (construction) network performs learning from scratch with a batch size of 128 using the ADAM optimizer, and all weights have a Gaussian omitted distribution with a mean of '0' and a standard deviation of '1' ( truncated Gaussian distribution).

8 is a flowchart illustrating a method of detecting defects through image construction based on artificial intelligence-based noise removal as an embodiment of the present invention.

On the other hand, as a method of the present invention, a defect detection method through image construction based on artificial intelligence-based noise removal is basically composed of a photographed image reception step, a noise removal step, an image construction step, and a defect detection and discrimination step. .

More specifically, referring to FIG. 8 , a captured image receiving step in which the reception unit receives a captured image generated by capturing an object, a noise removal step in which the defect detection and determination unit removes noise included in the captured image, and the noise removed An image construction step of constructing an image composed of only normal elements after removing bad elements in the image, and a defect detection and discrimination step of performing defect detection and discrimination through comparison with an image from which only noise has been removed based on the constructed image. It is done.

In the above description of the present invention with reference to the accompanying drawings, the specific shape and direction have been mainly described, but the present invention can be variously modified and changed by a person having ordinary knowledge in the technical field belonging to the invention, and these Modifications and changes should be construed as being included in the scope of the present invention.

Claims (11)

a receiving unit for receiving a photographed image generated by photographing an object; and
A defect detection and determination unit configured to determine object defect detection by removing noise included in the captured image, removing defective elements in the noise-removed image, and constructing an image composed of only normal elements;
The defect detection and discrimination unit,
Characterized in that it includes an artificial intelligence-based learning model for removing noise from the captured image and constructing an image composed of only normal elements.
Defect detection system through image construction based on artificial intelligence-based noise removal.
The method of claim 1,
The defect detection and discrimination unit,
Characterized in that the artificial intelligence-based learning model is configured based on deep learning,
Defect detection system through image construction based on artificial intelligence-based noise removal.
The method of claim 1,
The defect detection and discrimination unit,
Characterized in that additionally performing image acquisition consisting of a consistent form of color based on the denoised image,
Defect detection system through image construction based on artificial intelligence-based noise removal.
The method of claim 1,
The defect detection and discrimination unit,
Among the artificial intelligence-based learning models, including a noise removal model,
The denoising model is characterized in that it learns the features of the captured image and performs denoising by determining a part excluding the learned features as noise.
Defect detection system through image construction based on artificial intelligence-based noise removal.
The method of claim 4,
The noise removal model,
Characterized in that it includes an encoder configured with a structure in which a deep learning-based convolutional neural network (CNN) is continuously connected and has a final output value as all connected layers (Dense Layer),
Defect detection system through image construction based on artificial intelligence-based noise removal.
The method of claim 4,
The noise removal model,
Characterized in that it includes a decoder configured with a structure in which a deep learning-based data expandable convolutional neural network (Dilated CNN) is continuously connected and does not have a final output value as all connected layers (Dense Layer),
Defect detection system through image construction based on artificial intelligence-based noise removal.
The method of claim 1,
The defect detection and discrimination unit,
Among the artificial intelligence-based learning models, including a poor cognitive model,
The defect recognition model finds and removes defective elements based on the denoised image, builds an image with only normal elements, and performs defect detection and discrimination through comparison with the image from which only the noise has been removed.
Defect detection system through image construction based on artificial intelligence-based noise removal.
The method of claim 7,
The defect recognition model,
Characterized in that the deep learning-based convolutional neural network (CNN) includes a layer (reduction layer) that is continuously connected and maintains the dimension of the input data as it is,
Defect detection system through image construction based on artificial intelligence-based noise removal.
The method of claim 7,
The defect recognition model,
Characterized in that a deep learning-based convolutional neural network (CNN) is continuously connected and includes a feature extract layer in which only features are extracted without changing the data dimension,
Defect detection system through image construction based on artificial intelligence-based noise removal.
The method of claim 7,
The defect recognition model,
Characterized in that a deep learning-based data scalable convolutional neural network (Dilated CNN) is continuously connected and includes an upscaling layer that outputs data equal to the input size,
Defect detection system through image construction based on artificial intelligence-based noise removal.
In the defect detection method using the system according to any one of claims 1 to 10,
a photographed image reception step of receiving a photographed image generated by photographing an object by the receiving unit; and
and a defect detection and discrimination step in which the defect detection and discrimination unit removes noise included in the captured image, removes defective elements in the noise-removed image, and constructs an image composed of only normal elements to determine object defect detection. to do,
A defect detection method through image construction based on artificial intelligence-based noise removal.

Images