KR20220016245A - Apparatus and method for generating a defect image - Google Patents

Abstract

According to an embodiment of the present invention, provided is a method for generating a defective image performed by a device for generating a defective image, which includes the steps of: receiving a masking image in which a normal image and a defective shape are masked; removing the masked defective shape of the masking image from the normal image; and generating a defective image by inputting a normal image from which the defective shape is removed to a trained artificial neural network for generating the defective image.

Description

APPARATUS AND METHOD FOR GENERATING A DEFECT IMAGE

The present invention relates to an apparatus and method for generating a defective image image. More particularly, it relates to a defective image generating apparatus and method for generating a defective image necessary for detecting a defect in a product.

Recently, in the process of manufacturing various products, defects may occur on the surface of the product for various reasons. Among these products, various defects may occur on the surface, such as a display panel. For example, defects may occur due to various reasons, such as malfunction of a manufacturing machine during the manufacturing process of the panel, a manufacturer's mistake, dust or dust, or a defect in a film.

In relation to this, in Korean Patent Registration No. 10-1188404 in the prior art document, an image using the polarization characteristics of the display panel glass is acquired, Describe the defect inspection method. In the prior art literature, only the content of determining whether a glass image is defective by comparing the measured value with the threshold value of the image element of the glass image is described. However, the defect determination in the same manner as in the prior art document requires a lot of time and processing steps. In order to prevent this, there are increasing attempts to use artificial intelligence to inspect defects occurring in products. However, for inspection using artificial intelligence, defective data, that is, a defective image, is required for detecting defects.

However, since defects do not occur in a specific form, there is a problem in that there is a limitation in acquiring a large number of defective images. In addition, although a sufficient number of defective images are required to improve the performance of defect detection, there is a problem in that there is a limit to the acquisition of defective images.

Therefore, there is a need for a technique for solving the above-mentioned problems.

On the other hand, the above-mentioned background art is technical information that the inventor possessed for the purpose of derivation of the present invention or acquired during the derivation process of the present invention, and it cannot be said that it is necessarily known technology disclosed to the general public before the filing of the present invention. .

An object of the embodiments disclosed herein is to provide an apparatus and method for generating a defect image capable of acquiring a plurality of defective images including various defects.

SUMMARY Embodiments disclosed herein have an object to provide an apparatus and method for generating a defective image capable of securing a sufficient number of defective images to increase defect detection performance.

As a technical means for achieving the above-described technical problem, according to an embodiment, a method for generating a defective image performed by an apparatus for generating a defective image includes: receiving a masking image in the form of a binary image in which a normal image and a defective shape are masked , removing the masked bad shape of the masking image from the normal image, and generating a bad image by inputting the normal image from which the bad shape is removed into a trained artificial neural network to generate a bad image.

According to another embodiment, the defective image generating apparatus includes a normal image processor configured to receive a masking image in the form of a binary image in which a normal image and a defective shape are masked, and remove the masked defective shape of the masking image from the normal image, and and a bad image generator for generating a bad image by inputting a normal image from which the bad shape is removed to a trained artificial neural network to generate a bad image.

According to another embodiment, there is provided a computer program stored in a readable recording medium for performing a method for generating a bad image, which is performed by a game equipment recommendation device, wherein the method for generating a bad image includes a binary masking a normal image and a bad shape. Receiving a masking image in the form of an image, removing the masked bad shape of the masking image from the normal image, and inputting the normal image from which the bad shape is removed into a trained artificial neural network to generate a bad image generating an image.

According to another embodiment, there is provided a computer-readable recording medium on which a program for performing a method for generating a defective image is recorded, the method comprising: receiving a masking image in the form of a binary image in which a normal image and a defective shape are masked; , removing the masked bad shape of the masking image from the normal image, and generating a bad image by inputting the normal image from which the bad shape is removed into a trained artificial neural network to generate a bad image.

According to any one of the above-described problem solving means of the present invention, it is possible to provide an apparatus and method for generating a defect image capable of acquiring a plurality of defective images including various defects.

In addition, according to any one of the problem solving means of the present invention, it is possible to provide an apparatus and method for generating a defective image capable of securing a sufficient number of defective images to improve the performance of detecting the defect.

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description. will be.

1 is a block diagram illustrating an apparatus for generating a defective image according to an exemplary embodiment.
FIG. 2 is a diagram for explaining an operation of learning a bad image in an apparatus for generating a bad image according to an exemplary embodiment.
3 is a diagram for explaining an operation of learning a bad image in the device for generating a bad image according to an exemplary embodiment.
4 is a flowchart illustrating a method for learning a bad image performed by an apparatus for generating a bad image according to an exemplary embodiment.
5 is a flowchart illustrating a method for generating a defective image performed by an apparatus for generating a defective image according to an exemplary embodiment.
6 is a block diagram illustrating a failure determination apparatus according to an exemplary embodiment.

Hereinafter, various embodiments will be described in detail with reference to the accompanying drawings. The embodiments described below may be modified and implemented in various different forms. In order to more clearly describe the characteristics of the embodiments, detailed descriptions of matters widely known to those of ordinary skill in the art to which the following embodiments belong are omitted. In addition, in the drawings, parts irrelevant to the description of the embodiments are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a component is said to be “connected” with another component, it includes not only a case of 'directly connected' but also a case of 'connected with another component interposed therebetween'. In addition, when a component "includes" a component, it means that other components may be further included, rather than excluding other components, unless otherwise stated.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating an apparatus for generating a defective image according to an exemplary embodiment.

As shown in FIG. 1 , the defective image generating apparatus 100 may include a detector 110 , a reject remover 120 , a defective image generator 130 , and a normal image processor 140 .

The detector 110 may receive a defective sample image. Here, the defective sample image is an image obtained by directly photographing a sample having defects (eg, including defects) using a camera or the like. The detector 110 may detect a defective portion in the defective sample image. For example, the detector 110 may store a normal image, etc., and may detect a defective portion using a comparison between an input defective sample image and a pixel. The masking image generated by the detector 110 may be a binary image, and a defective part is displayed.

The detector 110 may generate a masking image indicating the defective portion by masking the defective portion in the defective sample image. The detector 110 may output the generated masking image to the defect remover 120 .

The defect remover 120 may receive a defective sample image and a masking image. The defect remover 120 may generate a defect removal image by matching the defective sample image and the masking image. For example, the reject remover 120 may multiply data values between pixels corresponding to each other in the defective sample image and the masking image to generate the reject removal image. In this way, the defect remover 120 may generate a defect removal image in which a defective portion is removed from the defective sample image.

The defective image generator 130 may receive a defective sample image and a defective removal image. The defective image generator 130 may learn a defect included in the defective sample image by using the defective sample image and the defect removal image.

The bad image generator 130 may be implemented using an artificial neural network for learning of a bad (ie, a defect). Here, the artificial neural network may include a convolutional neural network (CNN).

In this way, the bad image generator 130 may learn the bad image. When learning is completed, the bad image generator 130 may generate various bad images using an artificial neural network.

Next, a configuration for generating a bad image in the bad image generator 130 will be described.

The normal image processor 140 may receive a masking image obtained by masking the defective shape. Here, the masking image may be a binary image, and a defective shape for generating a defective image is displayed. The normal image processor 140 may receive a normal image, that is, a sample image. When it is possible to directly generate a masking image in the defective image generating apparatus 100, the defective image generating apparatus 100 is located in front of the normal image processor 140 and generates a masking image generating unit (not shown). It may include additionally.

The normal image processor 140 may generate an image from which the masking area is removed by matching the masking image to the normal image. Through this, the normal image processor 140 may reflect the defective shape of the masking image in the normal image. The normal image processor 140 may output the normal image from which the defective shape is removed to the defective image generator 130 .

The defective image generator 130 may receive a normal image from which the defective shape has been removed, and may generate the defective image using an artificial neural network. The bad image generator 130 may output the generated bad image.

On the other hand, since the masking image input to the normal image processor 140 can be simply generated, it can be easily created in various forms. Therefore, the bad image generator 130 implemented using the artificial neural network learned from the bad image receives the normal image in which the region corresponding to the masking region of the masking image is removed to generate a bad image close to the actual bad image. can

The bad image generator 130 may generate a bad image from a normal image in which a bad part is masked by using an artificial neural network trained with the bad images.

The bad image generator 130 may include an artificial neural network that can generate a bad image by receiving a normal image, and the artificial neural network uses, for example, an in-painting technique or blending to generate a bad image. ) techniques and generative adversarial networks (GANs) can be used.

For example, when the inpainting technique is used, the defective image generator 130 may learn a defective pattern corresponding to the masking area and fill in the defective portion in a natural form to generate a defective image. That is, the defective image generator 130 may reconstruct and fill in the masking area in the normal image with information related to the defect by using the infating technique.

For example, when the GAN technique is used, the bad image generator 130 may use the structure of a generator that generates a new bad image and a discriminator that evaluates the authenticity of the bad image generated by the generator. . By competitively learning the generator and discriminator, the bad image created by the generator can be generated in a form close to the actual bad image.

As described above, the defective image generating apparatus 100 may generate various defective images required for detecting the defective image.

The bad image generating apparatus 100 may use the detector 110 , the bad remover 120 , the bad image generator 130 , and the normal image processor 140 to learn the bad image, and after the learning is completed, When generating a bad image, the bad image generator 130 and the normal image processor 140 may be used.

Accordingly, the defective image generating apparatus 100 may be implemented with only the defective image generator 130 and the normal image processor 140 when the learned defective image generator 130 is used.

The defective image generating apparatus 100 or the defective image generator 130 may include an artificial neural network or may be implemented using an artificial neural network, and when learning using the defective image is completed, the defective image generator 130 inspects the defect of the product. It may be applied to and used to generate a defective image for detecting a defect.

At least some of the components constituting the defective image generating apparatus 100 may be implemented through a processor, for example, a control unit having a processor function such as a central processing unit (CPU).

FIG. 2 is a diagram for explaining an operation of learning a bad image in an apparatus for generating a bad image according to an exemplary embodiment.

As shown in FIG. 2 , the defective image generating apparatus 100 may receive a defective sample image 210 . The defective sample image 210 may include a defective portion 211 .

The defective image generating apparatus 100 may obtain the masking image 220 by masking the defective portion 211 in the defective sample image 210 . The masking image 220 is a binary image, and a defective part may be displayed in the masking image 220 .

The defective sample image may include defects that may occur during production or manufacturing, and may include defects that may occur during use. For example, the defective sample image may be formed to have a predetermined direction, may include a line-shaped defect having a predetermined length and thickness, and may be variously classified according to the length, thickness, and direction of the defect. The defective sample image may contain defects in a predetermined area or dot shape, and may be variously classified according to size and shape, and formed in the corners of the image (upper left, upper right, lower left, lower right, etc.) may contain defects. In addition, the defective sample image is a defect in the form of light spreading due to the phenomenon of light spreading, defects that occur in clusters in the form of clouds, defects in which layers are formed in the form of contour lines, and defects in non-uniform shapes that cannot confirm uniform rules or patterns, etc. may contain a variety of

The defective image generating apparatus 100 may obtain a defective removal image 230 in which the defective portion 211 is removed from the defective sample image by matching the defective sample image 210 and the masking image 220 with each other. The defective portion 231 is removed from the defective removal image 230 .

The defective image generating apparatus 100 may include the artificial neural network 10 , and may train the artificial neural network 10 using the defective sample image 210 and the reject removal image 230 . Through this, the artificial neural network 10 may learn the defective part 211 of the defective sample image 210 .

FIG. 3 is a diagram for explaining an operation of learning a bad image in an apparatus for generating a bad image according to an exemplary embodiment.

As shown in FIG. 3 , the defective image generating apparatus 100 may receive a normal image 310 and a masking image 320 in which the defective shape 321 is masked.

The defective image generating apparatus 100 may remove the masked defective shape 321 of the masking image 320 from the normal image 310 . The defective image generating apparatus 100 may generate the defective image 340 by inputting the normal image 330 from which the defective shape 331 is removed to the artificial neural network 10 .

Through this, the defective image generating apparatus 100 may output the defective image 340 in which the defective shape 341 is formed by inputting a normal image using the artificial neural network 10 .

4 is a flowchart illustrating a method for learning a bad image performed by an apparatus for generating a bad image according to an exemplary embodiment.

The method for learning a bad image according to the embodiment shown in FIG. 4 includes steps that are time-series processed by the device 100 for generating a bad image shown in FIG. 1 . Therefore, even if omitted below, the description above with respect to the apparatus 100 for generating a bad image shown in FIG. 1 may also be applied to the method for learning a bad image according to the embodiment shown in FIG. 4 .

As shown in FIG. 4 , the defective image generating apparatus 100 receives a defective sample image ( S410 ). The defective sample image may include defects of various shapes.

The defective image generating apparatus 100 may obtain a masking image obtained by masking the defective portion ( S420 ). The defective image generating apparatus 100 may detect a defective portion in the defective sample image and mask the detected defective portion. Here, the masking image may be a binary image composed of 0 and 1, or may be implemented as a contrast (black and white) image. For example, in the masking image, a defective portion may be displayed in black, and a portion other than the defective portion may be displayed in white.

The defective image generating apparatus 100 may obtain a defect removal image by matching the defective sample image and the masking image (S430). The defective image generating apparatus 100 may obtain a defect removal image in which a defective portion is removed from the defective sample image by using a masking image.

The defective image generating apparatus 100 may train the artificial neural network by using the defective removal image and the defective sample image (S440). At this time, the artificial neural network learns the defective part of the defective sample image.

The defective image generating apparatus 100 may train the artificial neural network by repeatedly performing the operation described with reference to FIG. 4 for learning the artificial neural network using a predetermined number of defective sample images.

5 is a flowchart illustrating a method for generating a defective image performed by an apparatus for generating a defective image according to an exemplary embodiment.

The method for generating a defective image according to the embodiment illustrated in FIG. 5 includes steps that are time-series processed by the apparatus 100 for generating a defective image illustrated in FIG. 1 . Accordingly, even if omitted below, the description above with respect to the apparatus 100 for generating a defective image shown in FIG. 1 may also be applied to the method for generating a defective image according to the embodiment shown in FIG. 5 .

As shown in FIG. 5 , the defective image generating apparatus 100 may receive a normal image and a masking image obtained by masking a defective shape ( S510 ). Here, the masking image may be a binary image composed of 0 and 1, or may be implemented as a contrast (black and white) image. For example, in the masking image, a defective shape may be displayed in black, and parts other than the defective shape may be displayed in white.

The bad image generating apparatus 100 may remove the bad shape of the masking image from the normal image ( S520 ). Through this, the defective image generating apparatus 100 may generate a normal image from which the defective shape is removed.

The defective image generating apparatus 100 may generate a defective image by inputting the normal image from which the defective shape is removed to the artificial neural network ( S530 ).

6 is a block diagram illustrating an apparatus for determining a failure according to an exemplary embodiment.

As shown in FIG. 6 , the defect determination apparatus 600 may receive a sample image and determine whether a defect exists in the sample image.

The failure determination apparatus 600 may include an artificial neural network for failure determination. Upon receiving the sample image, the defect determination apparatus 600 may determine whether a defect exists in the sample image.

The defect determination apparatus 600 may output a determination result of determining whether a defect exists in the sample image. In this case, when it is possible to determine the type of the defect, the defect determination apparatus 600 may check the type of the defect and also output information on the type of the confirmed defect.

Meanwhile, the failure determination apparatus 600 may use the failure image generated by the failure image generating apparatus 100 for learning of an artificial neural network that determines failure. When the failure determination apparatus 600 is implemented to correct the defective image according to the determination of the failure, it may further include a failure correction unit for correcting the determined failure.

The defective image generating apparatus 100 may generate a defective image by receiving a normal image or a masking image corresponding to the sample image. The failure image generating apparatus 100 may generate a failure image for training the artificial neural network in the failure determination apparatus 600 and provide it to the failure determination apparatus 600 . The defective image generating apparatus 100 may provide a sufficient amount of defective images for training of the artificial neural network to the failure determining apparatus 600 .

The failure determination apparatus 600 may train an artificial neural network using the failure data. Through this, when the learning of the artificial neural network is completed, the failure determination apparatus 600 may determine the failure of the sample image using the artificial neural network.

Meanwhile, when an artificial neural network that has been trained in the apparatus 100 for generating a defective image can be used, the failure determination apparatus 600 may determine a failure by using the artificial neural network that has been trained in the apparatus 100 for generating a failure image.

The term '~ unit' used in this embodiment means software or hardware components such as field programmable gate array (FPGA) or ASIC, and '~ unit' performs certain roles. However, '-part' is not limited to software or hardware. '~' may be configured to reside on an addressable storage medium or may be configured to refresh one or more processors. Accordingly, as an example, '~' indicates components such as software components, object-oriented software components, class components, and task components, and processes, functions, properties, and procedures. , subroutines, segments of program patent code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables.

The functions provided in the components and '~ units' may be combined into a smaller number of elements and '~ units' or separated from additional components and '~ units'.

In addition, components and '~ units' may be implemented to regenerate one or more CPUs in a device.

Also, the method for generating a bad image according to an embodiment of the present invention may be implemented as a computer program (or computer program product) including instructions executable by a computer. The computer program includes programmable machine instructions processed by a processor, and may be implemented in a high-level programming language, an object-oriented programming language, an assembly language, or a machine language. . In addition, the computer program may be recorded in a tangible computer-readable recording medium (eg, a memory, a hard disk, a magnetic/optical medium, or a solid-state drive (SSD), etc.).

Accordingly, the method for generating a bad image according to an embodiment of the present invention may be implemented by executing the above-described computer program by a computing device. The computing device may include at least a portion of a processor, a memory, a storage device, a high-speed interface connected to the memory and the high-speed expansion port, and a low-speed interface connected to the low-speed bus and the storage device. Each of these components is connected to each other using various buses, and may be mounted on a common motherboard or in any other suitable manner.

Here, the processor may process a command within the computing device, such as, for example, to display graphic information for providing a graphic user interface (GUI) on an external input or output device, such as a display connected to a high-speed interface. Examples are instructions stored in memory or a storage device. In other embodiments, multiple processors and/or multiple buses may be used with multiple memories and types of memory as appropriate. In addition, the processor may be implemented as a chipset formed by chips including a plurality of independent analog and/or digital processors.

Memory also stores information within the computing device. As an example, the memory may be configured as a volatile memory unit or a set thereof. As another example, the memory may be configured as a non-volatile memory unit or a set thereof. The memory may also be another form of computer readable medium, such as, for example, a magnetic or optical disk.

In addition, the storage device may provide a large-capacity storage space to the computing device. A storage device may be a computer-readable medium or a component comprising such a medium, and may include, for example, devices or other components within a storage area network (SAN), a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory, or other semiconductor memory device or device array similar thereto.

The above description of the present invention is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

100: bad image generating device 110: detector
120: bad remover 130: bad image generator
140: normal image processor 600: bad judgment device

Claims (12)

A method for generating a defective image performed by an apparatus for generating a defective image, the method comprising:
receiving a masking image in the form of a binary image in which a normal image and a bad shape are masked;
generating an image from which a masking area is removed by matching a masking image to the normal image; and
and generating a bad image by inputting the image from which the masking region has been removed to a trained artificial neural network to generate a bad image,
The method for generating a defective image, characterized in that the masking image is implemented as a black-and-white (contrast) image so that defective parts are displayed in black and parts other than the defective parts are displayed in white. The method of claim 1,
Prior to receiving the masking image,
The method of generating a bad image further comprising the step of training the artificial neural network. 3. The method of claim 2,
The step of learning the artificial neural network comprises:
receiving a defective sample image;
obtaining a masking image by detecting a defective portion in the defective sample image;
obtaining a defective removal image obtained by removing a defective portion from the defective sample image by matching the defective sample image and the masking image; and
and learning the artificial neural network by using the defective sample image and the defective removal image. The method of claim 1,
The artificial neural network is
A convolutional neural network (CNN), a method of generating bad images. The method of claim 1,
The artificial neural network is
A method of generating a bad image implemented using one of an inpainting technique, a blending technique, and a productive adversarial neural network (GAN). a normal image processor for receiving a masking image in the form of a binary image obtained by masking a normal image and a defective shape, and generating an image from which a masking region is removed by matching the masking image to the normal image; and
A bad image generator for generating a bad image by inputting the image from which the masking region is removed to a trained artificial neural network for generating a bad image,
The apparatus for generating a defective image, characterized in that the masking image is implemented as a black-and-white (contrast) image so that a defective portion is displayed in black and a portion other than the defective portion is displayed in white. 7. The method of claim 6,
a detector that receives a defective sample image and detects a defective portion in the defective sample image to obtain a masking image; and
and a defect remover configured to match the defective sample image and the masking image with each other to obtain a defective removal image obtained by removing a defective portion from the defective sample image, and output the defective image generator to the defective image generator. 8. The method of claim 7,
The bad image generator,
A defective image generating apparatus for learning the artificial neural network by using the defective sample image and the defective removal image. 7. The method of claim 6,
The artificial neural network is
A rogue image generator that is a convolutional neural network (CNN). 7. The method of claim 6,
The artificial neural network is
An apparatus for generating a bad image implemented using one of an inpainting technique, a blending technique, and a productive adversarial neural network (GAN). A computer-readable recording medium in which a program for performing the method according to claim 1 is recorded. A computer program stored in a readable recording medium for performing the method according to claim 1 performed by the rogue image generating apparatus.

Images