KR20210122429A - Method and System for Artificial Intelligence based Quality Inspection in Manufacturing Process using Machine Vision Deep Learning - Google Patents

Abstract

Provided are a method and a system for automatic defect detection in cosmetic container printing manufacturing process based on AI using vision deep learning. The method of automatic defect detection in cosmetic container printing manufacturing process based on AI using vision deep learning according to the present invention is provided to generate an image of a cosmetic container, train a classifier to classify a good product and a defective product using the generated image of the cosmetic container, and determine the cosmetic container as a good product or a defective product using the trained classifier. Thereby, the system finds a characteristic value of data of the determined object on its own by learning, and therefore automatic defect detection in the cosmetic container based on AI becomes possible in an inspection area which is hard to be standardized and thus depends on manual inspection.

Description

Automatic defect detection method and system in AI-based cosmetic container printing manufacturing process using image deep learning {Method and System for Artificial Intelligence based Quality Inspection in Manufacturing Process using Machine Vision Deep Learning}

The present invention relates to manufacturing management technology, and more particularly, to an automatic defect detection method and system in an AI-based cosmetic container printing manufacturing process using image deep learning.

The detection of defects occurring in the existing cosmetic container printing manufacturing process is mostly done by an inspector visually checking the product at the end of the process. However, human error occurs due to various factors such as organization and environment in defective inspection by a visual inspector. Therefore, despite the formation of a separate QC organization, about 3% to 5% of the total production is delivered as defective due to human error. The loss of the domestic printing manufacturing industry due to defective cosmetic containers amounts to 35 billion won annually. Companies are trying to increase the number of inspectors or improve the precision of production facilities to reduce the defect rate, but the labor cost burden and investment in machinery facilities are reaching a limit.

Therefore, the accuracy is not consistent depending on the condition and judgment ability of the operator, and it takes a lot of time. Therefore, there is a need for a system and method capable of automatically detecting a defect in the printing state of a cosmetic container.

To overcome the problems of manual inspection, an automated vision-based inspection system called Automated Optical Inspection (AOI) or Machine Vision has been introduced. Machine vision systems have replaced manual inspectors in many areas, such as defect inspection and cosmetic container sorting. However, the machine vision inspection method is a design method with an engineer setting rules one by one according to process characteristics and target items. Therefore, the accuracy of machine vision has a limitation in that performance depends on how well the engineer makes the rules, and when the analysis target is complex, the detection accuracy for fine defects is lowered. In particular, when the image is irregular and atypical, when it is difficult for an engineer to define a defective feature vector, the accuracy of equipment by machine vision is low, so it still relies on human visual inspection.

In general, image processing techniques used in machine vision enable high accuracy and high defect rate detection in a controlled situation where there is no change in the target item and the external factors are completely the same. It has limitations in that it is necessary to try a new model learning or to redesign the discrimination algorithm for re-learning.

JP 2014-153560 A (published on August 25, 2014) KR 10-1711073 B1 (Registration Date 2017.02.22.) KR 10-2073162 B1 (Registration Date 2020.01.29.) JP 6653929 B1 (Registration Date 2020.01.31.)

The present invention has been devised to solve the above problems, and an object of the present invention is to provide a method and system for automatic defect detection in an AI-based cosmetic container printing manufacturing process using image deep learning.

An object of the present invention is to eliminate human error, minimize the cost of quality control, and promote the productivity of mechanical equipment through the development of an automatic defect inspection (ADI) system.

According to an embodiment of the present invention for achieving the above object, an automatic defect detection method in an AI-based cosmetic container printing manufacturing process using image deep learning includes: supplying and printing a cosmetic container; generating an image by photographing the cosmetic container passing by the rail; and constructing an unsupervised learning model using the captured image: stopping printing of the container when the container is abnormal through the unsupervised learning model.

And, the unsupervised learning model, capturing the image of the photographed cosmetic container and collecting data; extracting a keypoint score from the collected data through an autoencoder; learning a defect detection model with the extracted feature points; It may include; determining whether the cosmetic container belongs to the normal category.

In addition, the cosmetic container image generating step may include: slicing the cosmetic container image; and expanding the fragmented cosmetic container image.

Also, in the expansion step, the cosmetic container image may be expanded by using at least one of alignment, lighting effect conversion, and noise effect application to the cosmetic container image.

And, in the automatic defect detection method in the AI-based cosmetic container printing manufacturing process using image deep learning according to an embodiment of the present invention, the extraction of a keypoint score is a pretrained model through transfer learning. ) to fine-tuning; Learning the model through a valuation autoencoder (Valuational AutoEncoder) and data labeling (Data Labeling); It may include; extracting a region of interest from the photographed cosmetic container image.

In addition, the determining step may include: classifying an image acquired using a feature point detection algorithm into a normal image and a bad image through cos-similarity between images in the image; performing keypoint extraction and matching through comparison between different images; calculating a keypoint scoring (KPS) indicating the degree of similarity between each image; classifying each image by the calculated feature point score; Outputting the classification number and the probability value as a determination result; may be configured.

And, the automatic defect detection method in the AI-based cosmetic container printing manufacturing process using image deep learning according to an embodiment of the present invention, collecting images of defective images and re-learning; may further include.

In addition, the re-learning step may be performed by a remote server.

On the other hand, according to another embodiment of the present invention, an automatic defect detection system in the AI-based cosmetic container printing manufacturing process using image deep learning, a camera generating an image of the cosmetic container; and a processor for learning a classifier for classifying a good product and a defective product from the generated cosmetic container image, and determining the cosmetic container as a good product or a defective product using the learned classifier.

As described above, according to the embodiments of the present invention, unlike the existing machine vision inspection technique, the automatic defect detection system in the AI-based cosmetic container printing manufacturing process using image deep learning uses Transfer Learning and Variational The Auto Encoder technology can find the characteristic value of the target data by itself by detecting the characteristic points, and since it has excellent field applicability and great differentiation, it is difficult to formulate defects, so machine vision-based inspection can be performed even in inspection areas that rely on manual inspection. it becomes possible

In addition, according to embodiments of the present invention, it is possible to eliminate human error, minimize the cost of quality control, and promote the productivity of mechanical equipment through the automatic defect detection system.

In addition, according to embodiments of the present invention, even when frequent change of the cosmetic container is required, it is possible to provide an automated learning and optimization method, thereby reducing the introduction cost.

1 is a view for explaining a process from supplying a cosmetic container according to the present invention to extracting an image
2 is an explanatory diagram of a defect inspection based on feature point detection according to the present invention;
3 is a flowchart provided for the explanation of the automatic defect detection process in the AI-based cosmetic container printing manufacturing process using image deep learning according to the present invention;
4 shows a process of constructing an unsupervised learning model according to the present invention.
5 is a block diagram of a system for performing an AI-based cosmetic container printing manufacturing process using image deep learning.
6 is a view for explaining an example of a feature point detection heat map for a sample image.
7 is a diagram showing an example of an inspection area setting screen designed by the defect selection S/W of the present invention.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

1. Feature point detection method using deep learning

Deep learning is a technology used to cluster or classify objects or data. A computer cannot distinguish a dog from a cat by just a picture, but a human can very easily tell the difference. To this end, a method called ‘Machine Learning’ was devised, which is a technology that inputs a lot of data into a computer and classifies similar ones. When a picture similar to a stored picture of a dog is input, the computer classifies it as a picture of a dog. Many machine learning algorithms have already appeared on how to classify data, but deep learning is a machine learning method proposed to overcome the limitations of artificial neural networks.

Deep learning learns and solves the joint weight problem with a network created by stacking layers between inputs and outputs by creating a deep neural network by mimicking a human neural network. The deep learning used here shows strength in the high-level abstraction of the object, which was the weakest in the existing machine learning techniques, through a combination of several non-linear transformation techniques.

In an embodiment of the present invention, an automatic defect detection method in the AI-based cosmetic container printing manufacturing process using image deep learning is presented. An automated vision inspection method that uses deep learning techniques to learn and automatically identify good and bad data of objects to be detected in classifying production cosmetic containers produced in manufacturing plants or inspecting and detecting external defects am.

In the quality inspection method according to an embodiment of the present invention, an optimized discrimination model is automatically generated by inputting learning data (cosmetic container image and good/defective). can

In addition, in the quality inspection method according to an embodiment of the present invention, it is possible to show strength in changes in external environmental factors by learning by including noise in input data, and re-learning and replacement of detection models through online user feedback information are possible.

In an embodiment of the present invention, ResNet (CVPR 2016) and DenseNet (CVPR 2017), which were published as existing image learning models, are utilized as image analysis methods for defect detection, 1) supply cosmetic containers to print, 2) rail An image is created by photographing the cosmetic container passing by, 3) an unsupervised learning model is built using the captured image, and 4) if the container is abnormal through the unsupervised learning model, container printing is stopped. Each will be described in detail below.

1) Printing by supplying cosmetic containers

1 is a diagram illustrating a process from supplying a cosmetic container to extracting an image. FIG. 1A is a device for supplying a cosmetic container, FIG. 1B is a picture of a printing device for printing, and FIG. A camera device for generating an image by photographing a cosmetic container, and FIG. 1D is a diagram illustrating an example of the generated image.

The cosmetic container was supplied through FIGS. 1A and 1B and cut through the process of printing.

2) Create an image by photographing the cosmetic container passing by the rail

As shown in FIG. 1C, an image is generated by photographing a cosmetic container in the process of being transported on the rail.

Here, the generated image can be processed in real time by performing automatic object detection by the camera at regular time intervals, and applying a tracking technology to the detected object using a deep learning network, and minimizes the area where the object is searched. method can be applied to design and train a deep network.

Here, the automatically generated image may include pictures, letters, and the like of the cosmetic container. Specifically, the processor detects the area of the cosmetic container by utilizing the deep learning-based network, and detects the printing area by utilizing the deep network within the detected area of the container. In this case, may output location information, size information, and a detection reliability value of each detected object.

Furthermore, since the processor does not know the size of the container in advance, the presence or absence of the object can be checked while searching the object area at each of the preset scales in the image.

In the detection of a container in a real environment, not only the accuracy of detection performance but also the speed of detection are very important. For example, the speed of a container transported on a rail is an important factor in determining the printing speed, and in a transport environment, it is necessary to provide various characteristics of the detected container, rather than simply detecting it.

3) Build an unsupervised learning model using the captured image

In order to identify inspections based on deep learning algorithms, it is necessary to build an unsupervised learning model of images that a computer creates feature expressions by itself through data learning. is learned

FIG. 4 is a diagram provided to explain a learning data generation process. In order to improve classification accuracy when discriminating complex image images and learn detailed label information, as shown in FIG. 4 , first, an image of a photographed cosmetic container is captured and data is collected.

A keypoint score is extracted from the collected data through an autoencoder as shown in FIG. 2 . Analysis of classifying each image by calculating a keypoint scoring (hereafter kps) indicating the degree of similarity between each image from a keypoint detection algorithm that extracts and matches keypoints through comparison between different images proceed with

In addition, in order to increase the operation speed by reducing the image analysis area, a template matching algorithm is used to designate areas with a possibility of defects and automatically finds the corresponding area.

As shown in S30c of FIG. 4 , the defect detection model is trained using the extracted feature points.

In the present invention, scenarios including professional techniques such as data labeling, image preprocessing, and learning model setting are excluded as much as possible, and the user level such as setting the inspection area using the inspection range setting (Crop) as shown in FIG. Configured /UX design. Here, labeling information including image index and defect information along with the image was used as learning data for deep learning. By generating each applied data, the number of training data can be increased. This not only reinforces noise according to environmental changes, but also has the advantage that the convolutional neural network model is used for data set expansion in order to collect sufficient data because it requires a lot of data for training as it is complex.

If the defect detection model is trained with the extracted feature points, a step of determining whether the cosmetic container belongs to the normal category is performed as shown in S30c of FIG. 4 .

To make a deep learning-based checker, a classifier is determined through optimization based on a large amount of training data generated through the learning process. Data used for learning is learned by using index and defect information as classification values along with image data.

In order to judge the printing status of cosmetic containers as good or bad, the image of the container is checked with the highest probability in the index, whether it is good or defective, and the judgment result is displayed. In this case, in the case of a defective cosmetic container, it may be determined to be defective and an area may be indicated.

4) If the container is abnormal through the unsupervised learning model, stop printing the container.

Information that the container is determined to be abnormal through the unsupervised learning model and misidentified through the HMI (Human-Machine Interface) device used by workers in the factory ), the corresponding image data and label information of the corresponding image data are delivered to the user, and the printing of the container is stopped. The operator takes action on the abnormal part of the printing device based on the information displayed on the HMI and starts printing again.

The training model of the deep learning algorithm requires a significant amount of computation and training time to optimize the weights, and in particular, requires a lot of GPU (Graphics Processing Unit) computation.

Therefore, in a server equipped with a high-performance GPU at a remote location, error data separately collected as user feedback information is used as training data to create a new discrimination model.

On the other hand, in a situation in which the vision inspection device is used as an inspection device in the production process, a new type of defect that has not been learned before occurs or an error occurs in the previously learned discrimination model. By using additional data, an improved model can be created by the following procedure.

2. Machine vision-based quality inspection system

5 is a block diagram of an automatic defect detection system in an AI-based cosmetic container printing manufacturing process using image deep learning. The illustrated machine vision-based quality inspection system is a computing system including a camera 310 , a communication unit 320 , a processor 330 , a storage unit 340 , and a display 350 .

The camera 310 is an image generating device for generating an image by photographing a cosmetic container to be inspected.

The processor 330 extracts a region of interest from the image generated by the camera 310 and performs image processing, and is a means for generating and driving a classifier by learning and driving, and includes a CPU and GCUs.

The storage unit 340 provides a storage space necessary for the processor 330 to perform the above procedures.

The display 350 displays whether or not a defect is determined according to the cosmetic container inspection and a determination value.

The communication unit 320 is a means for online-based re-learning, and is a means for receiving a model improved by the server through re-learning.

3. Variations

So far, preferred embodiments of the automatic defect detection method and system in the AI-based cosmetic container printing manufacturing process using image deep learning have been described in detail.

In an embodiment of the present invention, a deep learning algorithm is applied to classify products and identify defects in a manufacturing plant, and through deep learning, a defect detection model is automatically created through image data learning to be atypical rather than a simple pattern. It made it possible to analyze even complex video images.

In addition, in embodiments of the present invention, a detection model can be created automatically by learning data, and self-optimization through online learning through user feedback is possible, thereby providing high flexibility, so that the vision inspection system It can lower the cost of introduction and change.

On the other hand, it goes without saying that the technical idea of the present invention can also be applied to a computer-readable recording medium containing a computer program for performing the functions of the apparatus and method according to the present embodiment. In addition, the technical ideas according to various embodiments of the present invention may be implemented in the form of computer-readable codes recorded on a computer-readable recording medium. The computer-readable recording medium may be any data storage device readable by the computer and capable of storing data. For example, the computer-readable recording medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical disk, hard disk drive, or the like. In addition, the computer-readable code or program stored in the computer-readable recording medium may be transmitted through a network connected between computers.

In addition, although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention as claimed in the claims Various modifications are possible by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or prospect of the present invention.

310: camera
320: communication department
330: processor
340: storage
350: display

Claims (9)

supplying and printing cosmetic containers;
generating an image by photographing the cosmetic container passing on the rail;
Building an unsupervised learning model using the generated image:
An automatic defect detection method in the AI-based cosmetic container printing manufacturing process using image deep learning, comprising: stopping the container printing if the container is abnormal through the unsupervised learning model. The method according to claim 1,
The unsupervised learning model is
capturing the image of the photographed cosmetic container and collecting data;
extracting a keypoint score from the collected data through an autoencoder;
learning a defect detection model with the extracted feature points;
Automatic defect detection method in the AI-based cosmetic container printing manufacturing process using image deep learning, characterized in that it comprises; determining whether the cosmetic container belongs to the normal category. The method according to claim 1,
The cosmetic container image creation step is,
slicing the cosmetic container image; and
Expanding the fragmented cosmetic container image; Automatic defect detection method in the AI-based cosmetic container printing manufacturing process using image deep learning, characterized in that it includes. 4. The method according to claim 3,
The expansion step is
An automatic defect detection method in the AI-based cosmetic container printing manufacturing process using image deep learning, characterized in that the cosmetic container image is expanded by using at least one of alignment, lighting effect conversion, and noise effect application to the cosmetic container image. 3. The method according to claim 2,
The extraction of the keypoint score is
Fine-tuning a pretrained model through transfer learning;
Learning the model through a valuation autoencoder (Valuational AutoEncoder) and data labeling (Data Labeling);
Extracting the region of interest from the photographed cosmetic container image; An automatic defect detection method in the AI-based cosmetic container printing manufacturing process using image deep learning, characterized in that it includes. 6. The method of claim 5,
The decision step is
classifying an image acquired using a feature point detection algorithm into a normal image and a bad image through cos-similarity between images in the image;
performing keypoint extraction and matching through comparison between different images;
calculating a keypoint scoring (KPS) indicating the degree of similarity between each image;
classifying each image by the calculated feature point score;
Outputting the classification number and probability value as a determination result; Automatic defect detection method in the AI-based cosmetic container printing manufacturing process using image deep learning, characterized in that it consists of. 7. The method of claim 6,
Automatic defect detection method in the AI-based cosmetic container printing manufacturing process using image deep learning, characterized in that it further comprises; collecting and re-learning the images of the defective image. 8. The method of claim 7,
The re-learning phase is
Automatic defect detection method in AI-based cosmetic container printing manufacturing process using image deep learning, characterized in that it is performed by a remote server. a camera for generating an image of a cosmetic container;
AI-based cosmetics using image deep learning, comprising: a processor that trains a classifier to classify good and bad products with the generated cosmetic container image, and determines a cosmetic container as good or bad product using the learned classifier Automatic defect detection system in the container printing manufacturing process.

Images