KR20210038211A - Method of inspection using image masking operation - Google Patents

Abstract

A defect inspection method using image masking is disclosed. A defect inspection method using image masking according to an embodiment includes an image photographing step of photographing an image of an object disposed on a work bench; a masking processing step of masking a predetermined area of the photographed image; and a defective product classification step of determining whether a product is defective by using an area other than the masked area and classifying the defective product.

Description

Defect inspection method using image masking {METHOD OF INSPECTION USING IMAGE MASKING OPERATION}

The present invention relates to a defect inspection method using image masking, and more particularly, to a defect inspection method using image masking to determine whether there is a defect after masking an unnecessary area in a photographed image.

Recently, with the advent of image sensors, digital cameras or portable terminals equipped with camera functions have become common, and as a result, digital photographs and images are commonly used by general consumers. In addition, many studies on the field of image recognition using cameras are active, and industrialization using them is steadily increasing.

For example, technologies such as fingerprint recognition, face recognition, iris recognition, text recognition, and object recognition related to security are such. These technologies are technically based on image recognition using a digital camera (hereinafter referred to as'Machine Vision' based), and in a machine vision-based device, the camera may be a hardware configuration corresponding to the human eye. Other components included to recognize an image generated through the camera may be hardware (or software) corresponding to the brain. Particularly, among the above-described machine vision-based technologies, object recognition technology is a technology that recognizes the number, shape, and size of objects captured in image data, and is used in various industrial fields.

Such object recognition technology can be used for character recognition, component recognition of factory assembly lines, defect inspection of PCB boards, and defective products by comparing object types as needed for defective or passed inspection in factory automation devices or automatic inspection devices. Or, an image processing technology for classifying passed products is being used.

Prior art Korean Patent No. 10-1017323 discloses an image processing apparatus and method through real-time subject shape comparison, and Korean Patent No. 10-0779918 discloses an image processing apparatus for stereo vision inspection and an inspection method using the same. I'm doing it.

However, even for the same object, since shape characteristics of a captured image may be detected differently according to a location, size, and arrangement angle, there is a problem in that the accuracy of determining the similarity in shape between objects is degraded.

This research was conducted by the Ministry of Science, ICT and ICT support for the ICT/SW convergence support project for the machinery industry.

The present invention was conceived to solve the problems of the prior art as described above, by taking an image of an object and determining whether a part is defective by comparing it with pre-stored data, thereby automating the existing manual operation. Its purpose is to provide a defect inspection method using image masking that can save labor costs for work.

In addition, the present invention provides a defect inspection method using image masking that can improve reliability of work while minimizing the amount of computation required for image processing by classifying areas of a photographed image and masking unnecessary areas. There is a purpose

In addition, the present invention determines whether there is a defect by comparing the reference data for determining whether there is a defect with the actually photographed image, but by repeatedly learning the process of determining whether the defect is defective, the accuracy of the determination of the defect is improved and included in the reference data. Its purpose is to provide a defect inspection method using image masking that can minimize errors by continuously updating the contents.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems to be solved by the present invention not mentioned here are to those of ordinary skill in the technical field to which the present invention belongs from the following description. It can be clearly understood.

The defect inspection method using image masking according to the present invention includes an image capturing step of capturing an image of an object disposed on a work table, a masking processing step of masking a preset area of the captured image, and an area excluding the masked area. And a defective product classification step of determining whether there is a defect using and classifying the defective product.

In addition, in the defect inspection method using image masking according to the present invention, the masking processing step includes setting at least one reference point on the photographed image, extracting at least one feature point of an object from the photographed image, and the feature point. And performing a mask processing operation on an unnecessary area based on the mask processing operation, wherein the mask processing operation is performed with at least one of a mask mask, a guide mask, and a complex mask in which the mask mask and the guide mask are combined.

In addition, in the defect inspection method using image masking according to the present invention, in the defective product classification step, the masked image is compared based on a previously stored codebook, and it is determined whether the object is defective, but the codebook is It is prepared by storing the image as a sample image, extracting at least one defect specific point from the stored sample image, formulating and classifying the extracted defect characteristic point, and performing the step of generating a code, and derived from the defective product classification step Further comprising a machine learning step of learning the result, updating the code stored in the codebook, and minimizing an error between the defective feature point extracted from the sample image and the feature point extracted from the photographed image.

In addition, in the defect inspection method using image masking according to the present invention, in the machine learning step, the average square error of the result derived from the defective product classification step is used, and learning is performed through a weighted average value and a regression analysis method.

By means of solving the above problems, the defect inspection method using image masking of the present invention captures an image of an object and compares it with pre-stored data to determine whether a component is defective, thereby automating the existing manual operation. It has the effect of saving labor costs for work.

In addition, the defect inspection method using image masking of the present invention has the effect of improving the reliability of work while minimizing the amount of computation required for image processing by classifying areas of a photographed image and masking unnecessary areas. .

In addition, the defect inspection method using image masking of the present invention determines whether there is a defect by comparing the reference data for determining whether there is a defect and the actually photographed image, but by repeatedly learning the process of determining whether there is a defect, It has the effect of minimizing errors by improving accuracy and continuously updating the contents included in the reference data.

1 is a flowchart of a defect inspection method using image masking according to an embodiment of the present invention.
2 is a flowchart of a masking process step by a defect inspection method using image masking according to an embodiment of the present invention.
3 is a diagram for explaining a mass masking process step of a defect inspection method using image masking according to an embodiment of the present invention.
4 is a diagram illustrating a guide masking process step of a defect inspection method using image masking according to an embodiment of the present invention.
5 is a view for explaining a complex masking process step of a defect inspection method using image masking according to an embodiment of the present invention.
6 is a view for explaining a defect classification step by a defect inspection method using image masking according to an embodiment of the present invention.
7 is a flowchart illustrating a codebook for a defect inspection method using image masking according to an embodiment of the present invention.
8 is a diagram illustrating a codebook of a defect inspection method using image masking according to an embodiment of the present invention.
9 is a diagram illustrating a machine learning step of a defect inspection method using image masking according to an embodiment of the present invention.

Specific matters, including the problems to be solved, means for solving the problems, and effects of the present invention as described above, are included in the following examples and drawings. Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings.

Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings.

1 is a flowchart of a defect inspection method using image masking according to an embodiment of the present invention.

Referring to FIG. 1, the defect inspection method using image masking may include an image capturing step (S100 ), a masking processing step (S200 ), and a defective product classification step (S300 ).

More specifically, the image capturing step (S100) is a step of capturing an image of an object disposed on a work table, and the masking processing step (S200) is a step of masking a preset area of the captured image, The defective product classification step S300 may include determining whether there is a defect using an area excluding the masked area, and classifying the defective product.

In the image capturing step (S100), the photographing apparatus includes a machine vision camera, and the camera may photograph at least one of a front surface, a plane surface, or a side surface of the object.

In addition, the image capturing step (S100) may include a pre-processing process such as noise removal and scale adjustment.

2 is a flowchart of a masking process step by a defect inspection method using image masking according to an embodiment of the present invention.

Referring to FIG. 2, the masking processing step (S200) includes setting at least one reference point on the captured image (S210), and extracting at least one feature point of the object from the captured image (S220). ) And performing a mask processing operation on an unnecessary area based on the feature point (S230).

In addition, the mask processing operation may be performed with at least one of a mask mask, a guide mask, and a composite mask in which the mask mask and the guide mask are combined.

In more detail, since the photographed image cannot be photographed consistently due to the shaking of the photographing device or the product, a reference point is set on the photographed image, and the masking processing step (S200) is performed under the same conditions. I can.

For example, one pixel of the photographed image is set as a first reference point, a point at a predetermined position in the X-axis direction from the first reference point is a second reference point, and a preset point in the Y-axis direction from the first reference point. A point at the location may be set as a third reference point, and a point at which the second reference point and the third reference point meet each other may be set as the fourth reference point.

Meanwhile, the feature point may be extracted from the captured image, at least one or more extracted according to the shape of the object, and may be extracted in the form of a point or a surface.

<Example 1>

3 is a diagram for explaining a mass masking process step of a defect inspection method using image masking according to an embodiment of the present invention.

As an example, referring to FIG. 3, in the masking processing step (S200), in the step of setting at least one reference point on the captured image (S210), the photographed at a predetermined size based on the reference point 211a In the step of dividing an image and extracting at least one feature point of the object from the photographed image (S220), after analyzing the divided image to extract at least one feature point of the object, a region where defects frequently occur ( 221a to 223a) and the region 224a in which no defect occurs, and in the step (S230) of performing a mask processing operation for an unnecessary region based on the characteristic point, the region 224a in which the defect does not occur is defective. Since there is no need to proceed with the inspection, it can be masked.

<Example 2>

4 is a diagram illustrating a guide masking process step of a defect inspection method using image masking according to an embodiment of the present invention.

As an example, referring to FIG. 4, in the masking processing step (S200), the image captured based on the reference point 211b in the step (S210) of setting at least one reference point on the captured image is a preset size. After dividing into and extracting at least one feature point of the object from the photographed image (S220), after analyzing the divided image and extracting at least one feature point of the object, a region 221b where defects frequently occur ~223b), and in the step (S230) of performing a mask processing operation on an unnecessary area based on the feature point, a guide mask 231b may be applied to the areas 221b ~ 223b where the defect is frequently generated. have.

The step (S220) of extracting at least one feature point of the object from the photographed image may be omitted, and the guide mask 231b may be applied to the entire divided image.

In addition, the guide masking 231b may be applied in at least one of vertical, horizontal, grid, and oblique lines.

<Example 3>

5 is a view for explaining a complex masking processing method of a defect inspection method using image masking according to an embodiment of the present invention.

As an example, referring to FIG. 5, in the masking processing step (S200), in the step of setting at least one reference point on the captured image (S210), the image captured based on the reference point 211c is set to a preset size. In the step of dividing into and extracting at least one feature point of the object from the photographed image (S220), after analyzing the divided image to extract at least one feature point of the object, an area where defects frequently occur (221c) ~223c) and the area 224c where the defect does not occur, and in the step (S230) of performing a mask processing operation on the unnecessary area based on the characteristic point, the areas 221c ~ 223c where the defect occurs frequently The guide mask 231c may be applied to the area 224c, and the area 224c in which the defect does not occur may be subjected to a mask treatment.

6 is a view for explaining a defect classification step by a defect inspection method using image masking according to an embodiment of the present invention.

Referring to FIG. 6, in the step of classifying defective products (S300 ), the masked image may be compared based on a previously stored codebook, and whether or not defective may be determined.

For example, a template 321 from which the at least one defective feature point is extracted from the sample image of the object determined as the defective product and the masked image 230 are compared, but the similarity with the template 321 is determined. It is possible to calculate and determine whether the photographed object is defective based on the calculated similarity and a preset criterion (ex. defective determination when the similarity is 70% or more).

Meanwhile, FIG. 7 is a flowchart illustrating a codebook for a defect inspection method using image masking according to an embodiment of the present invention, and FIG. 8 is a codebook for a defect inspection method using image masking according to an embodiment of the present invention. It is a drawing to do.

Referring to FIG. 7, in the codebook, storing the image of the object determined as a defective product as a sample image (S310), extracting at least one defective feature point from the stored sample image (S320), and the extracted defect It may be prepared by formulating and classifying the feature points, and performing the step (S330) of generating a code.

The sample image is composed of images of defective products classified as defective, and may include all images of the defective products that may occur during the working process.

In addition, the defective feature point is extracted from the sample image of the defective product, and the extracted defective feature point may be formulated as shown in FIG. 8.

9 is a diagram illustrating a machine learning step of a defect inspection method using image masking according to an embodiment of the present invention.

9, by learning the result derived in the defective product classification step (S300), updating the code stored in the codebook, the defective feature extracted from the sample image and the feature point extracted from the photographed image, It may further include a machine learning step (S400) to minimize the error of.

In more detail, in the machine learning step (S400), the step of extracting at least one defective feature point from the stored sample image in the defective product classification step (S300) (S320) and the extracted defect feature point by formulating and classifying it. , The step of generating a code (S330) is performed, and the process of comparing the captured image and the sample image is repeatedly learned, and the repeatedly learned content may be reflected in the code stored in the codebook.

For example, assuming that the coil is not normally packaged on one side of the product, and the code'A001' is assigned to the codebook for a defect in which a part of the product is protruding, the mass mask is processed for the'A001' defect. The result that it takes less time to classify defects using the guide mask-processed image than the time required to classify defects using the image is repeatedly learned, and the'A001' For defects, the guide mask operation may be performed to classify the defects.

In this case, the machine learning step (S400) uses the average square error of the result derived in the defective product classification step (S300), and can be learned through a weighted average value and a regression analysis method.

According to the effects of the present invention as described above, by taking an image of an object and determining whether or not a component is defective by comparing it with previously stored data, it is possible to automate the existing manual operation and save the labor cost required for the operation. A defect inspection method using existing image masking may be provided.

In addition, a defect inspection method using image masking can be provided that can improve reliability of work while minimizing the amount of computation required for image processing by classifying areas of the captured image and masking unnecessary areas.

In addition, by comparing the reference data for determining whether there is a defect and the actually photographed image to determine whether it is defective or not, by repeating the process of determining whether or not it is defective, the accuracy of determining whether or not defective is improved and the contents included in the reference data A defect inspection method using image masking capable of minimizing an error by updating may be provided.

In addition, the defect inspection method using image masking according to an embodiment of the present invention may be recorded in a computer-readable medium including program instructions for performing operations implemented by various computers. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The medium may be a program command specially designed and configured for the present invention, or may be known to and usable by a person skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

As described above, although an embodiment of the present invention has been described by a limited embodiment and the drawings, an embodiment of the present invention is not limited to the above-described embodiment, which is a common knowledge in the field to which the present invention pertains. Anyone who has it can make various modifications and variations from these substrates. Therefore, one embodiment of the present invention should be grasped only by the claims described below, and all equivalent or equivalent modifications thereof will be said to belong to the scope of the inventive concept.

S100: Image capture stage
S200: Masking processing step
S300: Defective product classification step

Claims (4)

An image photographing step of photographing an image of an object disposed on the workbench;
A masking processing step of masking a predetermined area of the captured image; And
A defective product classification step of determining whether there is a defect using an area excluding the masked area and classifying the defective product;
Defect inspection method using image masking comprising a. The method of claim 1,
The masking processing step,
Setting at least one reference point on the captured image;
Extracting at least one feature point of the object from the captured image; And
Performing a mask processing operation on an unnecessary area based on the feature point;
Including,
The mask processing operation is performed with at least one of a mask mask, a guide mask, and a composite mask in which the mask mask and the guide mask are combined. The method of claim 1,
The step of classifying defective products,
Compare the masked image based on a previously stored codebook, and determine whether the object is defective,
The codebook,
Storing the image of the object determined as a defective product as a sample image;
Extracting at least one defect feature from the stored sample image; And
Prepared by performing the step of formulating and classifying the extracted defective feature points, and generating a code,
A machine learning step of learning a result derived from the defective product classification step, updating a code stored in the codebook, and minimizing an error between the defective feature point extracted from the sample image and the feature point extracted from the photographed image;
Defect inspection method using image masking further comprising a. The method of claim 3,
The machine learning step,
A defect inspection method using image masking, characterized in that using the mean square error of the result derived in the defective product classification step, and learning through a weighted average value and a regression analysis method.

Images