KR20200084444A - Deep learning based classification system using image data augmentation, and cotrol method thereof - Google Patents

Abstract

The present invention relates to a classification system using an image data generation technology based on deep learning and to a control method thereof. According to the present invention, the system comprises: an image data generation unit including an encoder compressing input image data and extracting a feature, and a decoder generating reconstructed image data reconstructing the input image data compressed by the encoder; and a convolutional neural network unit. A data classification portion for classifying input image data is constructed using the encoder and the convolutional neural network. According to the present invention, disadvantages of simply inputting deep learning as a defect image can be overcome, the defect classification performance is maximized, and the operation time can be minimized.

Description

Deep learning based classification system using image data augmentation, and cotrol method thereof}

The present invention relates to a classification system and a control method thereof, and more particularly, to a classification system using a deep learning-based image data generation technology and a control method thereof.

Recently, the interest in artificial intelligence has increased greatly due to the confrontation between AlphaGo and Lee Sedol 9th. In particular, academic and industrial research on deep learning, known as the core technology of AlphaGo, has exploded.

Deep learning is the development and dropout of ReLU (Rectified Linear Unit) for the problems of known artificial neural networks (ANN), that is, vanishing problems and overfitting. out) to improve performance. Also, thanks to advances in hardware such as GPUs (Graphic Processing Units) and the power of big data capable of learning complex structures, it has been showing outstanding performance in various fields in recent years.

This deep learning technology has been rapidly developed by many overseas companies (Google, Facebook, Apple, Microsoft, Alibaba, Baidu), and is applied to fields such as face recognition, voice recognition, natural language processing, search services, and medical care. . It is imperative to secure the latest technology in this fast-developing deep learning, and further preoccupy the application field and commercialize it quickly.

1 is a diagram illustrating a defect classification method used in the existing defect inspection equipment.

Referring to FIG. 1, in the defect classification method used in the existing defect inspection equipment, an algorithm developer extracts features that are likely to be classified from an image into an image processing algorithm, and then learns these features using a classifier (SVM, Decision tree). do. In this method, how well a person designs a feature extraction algorithm to extract features is a limitation of performance. On the other hand, the recent classification method using deep learning is that artificial intelligence extracts features and learns the classifier by minimizing human intervention.

Korean Registered Patent No. 1,863,196 (Registration Date: 2018.05.25) U.S. Patent Publication No. 2017-0200264 (published on: July 13, 2017)

Therefore, the technical problem to be solved by the present invention is to overcome the disadvantages of the case where the input of deep learning is simply a defect image and to maximize the defect classification performance while minimizing the computation time while classifying defects using deep learning-based image data generation technology. It is to provide a system and method.

An image classification system according to the present invention for solving the above technical problem includes an encoder for compressing input image data and extracting features and a decoder for generating reconstructed image data by restoring the input image data compressed by the encoder. It includes an image data generating unit and a convolutional neural network unit.

A data classification unit is configured to classify input image data using the encoder and the convolutional neural network unit.

The data classification unit may be trained using the learning data generated by the image data generation unit.

When learning the data classifying unit, the encoder may be separated from the decoder to be arranged in front of the convolutional neural network unit.

The image data generation unit may be an autoencoder.

The image data generation unit may be a CCVAE (Conditional Convolutional Variational Autoencoder).

The input image data may be image data obtained by photographing the surface of a product to be inspected by applying illumination to inspect for surface defects.

The control method of the image classification system according to the present invention for solving the above technical problem is an encoder for compressing input image data and extracting features and a decoder for generating reconstructed image data by restoring the input image data compressed by the encoder. And generating learning data by the image data generation unit including, constructing a data classification unit using the encoder and the convolutional neural network unit, and training the data classification unit using the generated learning data.

It may include a computer-readable recording medium containing a program for performing a control method of the image classification system according to the present invention for solving the above technical problem.

The program comprises: an instruction for generating training data by an image data generator including an encoder for compressing input image data and extracting features, and a decoder for generating reconstructed image data by restoring the input image data compressed by the encoder. A set, a set of instructions for constructing a data classifier using a convolutional neural network that classifies the encoder and input image data according to extracted features, and a set of instructions for training the data classifier using the generated training data Can be.

According to the present invention, deep learning can self-extract and learn the characteristics of a defect from an image. In particular, there is an advantage of overcoming the disadvantages of deep learning input only by using a defect image and maximizing defect classification performance while minimizing computation time.

1 is a diagram illustrating a defect classification method used in the existing defect inspection equipment.
2 is a view showing the connection of components in the operation of generating learning data in the video classification system according to the present invention.
3 is a view showing the connection of components when learning the data classification unit of the image classification system according to the present invention.
4 is a flowchart illustrating an operation process of the image classification system according to the present invention.

Then, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present invention pertains can easily practice.

2 is a view showing the connection of components in the operation of generating learning data in the video classification system according to the present invention.

First, the configuration of a general product defect inspection system will be described. A typical product defect inspection system consists of an optical device using lighting and a camera. The path of light in the defect area is changed to image the amount of light entering the camera to increase the signal noise ratio (S/N ratio) of the defect area. As described above, a defect detection algorithm detects a defect candidate from input image data obtained by photographing the surface of a product to be inspected by applying illumination to inspect for surface defects, and assigns a defect name by classifying the defect candidate image as a classification algorithm can do. And the system is structured by transmitting these results to the operation monitoring system and storing it in the DB. However, in order to secure excellent performance by applying deep learning technology to defect image classification, sufficient data must be secured for each defect. However, due to the nature of the manufacturing industry, it is difficult to obtain sufficient data for some defects, and the number of data collected for each defect may be large.

That is, in order to solve this problem, the image classification system 100 according to the present invention can generate sufficient learning data through the image data generation unit 110 as illustrated in FIG. 2.

The image data generator 110 may include an encoder 111, a sampling unit 113, and a decoder 115.

The encoder 111 compresses the input image data (x) and can learn by extracting a feature of the input image data. That is, the encoder 111 serves to extract characteristics of the high-dimensional input image data x and compress the high-dimensional input image data to a low dimension.

The sampling unit 113 serves to transfer the data z sampled with a specific probability from the data compressed by the encoder 111 to the decoder 115.

The decoder 115 plays a role of generating the reconstructed image data identical to the input image data, which has restored the input image data compressed by the encoder 111.

As described above, the function of generating the reconstructed image data obtained by restoring the input image data by the image data generation unit 110 may sufficiently secure insufficient learning data.

The image data generation unit 110 may be implemented by an auto encoder. The autoencoder may be a stacked auto encoder, a sparese auto encoder, or a denoising auto encoder. In addition, the autoencoder may be implemented with VAE (Variational Autoencoder), CAE (Contractive Autoencoder), CVAE (Conditional VAE), or CCVAE (Conditional Convolutional Variational Autoencoder).

In this way, the data classification unit that classifies the input data in the system 100 may be trained with the input image data (for example, the defect image data photographing the surface of the product) as the training data generated by the image data generation unit 110.

3 is a view showing the connection of components when learning the data classification unit of the image classification system according to the present invention.

Referring to FIG. 3, the system 100 according to the present invention may constitute a data classification unit 130 that classifies input image data by a convolutional neural network unit 120 and an encoder 111. To this end, when the system 100 learns the data classification unit 130, as illustrated in FIG. 3, the encoder 111 is separated from the decoder 115 to be disposed at the front end of the convolutional neural network unit 120. can do.

To this end, the system 100 may have a switching structure for selectively connecting the encoder 111 to the decoder 115 and the convolutional neural network unit 120.

The convolutional neural network unit 120 learns to classify input image data according to extracted features according to a convolution neural network (CNN) or a deep neural network (DNN). And so on.

In general, when learning a deep learning model for classifying a defect image, the classification performance greatly depends on which initial value of the classification model is selected. In general, when training a deep learning model for classification using trained data, a new classification model is learned. The classification model extracts a feature from the image itself, classifies the feature by a desired number of classes, and informs the defect of the final defect image.

The system 100 according to the present invention trains the data classification unit 130 with the data generated and learned by the image data generation unit 110. However, the encoder 111 serving as a feature extraction function in the image data generation unit 110 is also used as a structure for the feature extraction function in the data classification unit 130 and is used as an initial value. In addition, by arranging the convolutional neural network unit 120 for classification after the encoder 111, it is possible to provide a structure for quickly learning with only a much smaller layer than the conventional method. In addition, since the initial value is already used as the value learned by the image data generator 110 in the feature extraction layer, it can have much better classification performance.

4 is a flowchart illustrating an operation process of the image classification system according to the present invention.

2 to 4, first, the image data generation unit 110 including the encoder 111 and the decoder 115 may receive input image data and generate learning data (S410 ).

Next, after sufficient learning data is generated in step S410, a data classification unit may be constructed using the encoder 111 and the convolutional neural network unit 120 (S420). In step S420, the encoder 111 is separated from the decoder 115 so that it can be arranged at the front end of the composite neural network unit 120.

Then, the data classification unit 130 may be trained using the learning data generated in step S410 (S430).

Finally, the input image data may be classified using the data classification unit 130 learned in step S430 (S440).

The system and method according to the present invention described above can be used as a core technology for product defect inspection in various industries. The technique of classifying defects to analyze product defects can be used not only in surface defect inspection, but also in diagnostic fields such as non-destructive inspection. In addition, it is an essential technology for implementing a Smart Factory, and it can be applied to technologies that determine the quality of products according to production operation conditions and use them to optimize production conditions.

The embodiments described above may be implemented with hardware components, software components, and/or combinations of hardware components and software components. For example, the devices, methods, and components described in the embodiments include, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors (micro signal processors), microcomputers, and field programmable gates (FPGAs). It can be implemented using one or more general purpose computers or special purpose computers, such as arrays, programmable logic units (PLUs), microprocessors, or any other device capable of executing and responding to instructions. The processing device may run an operating system (OS) and one or more software applications running on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of understanding, a processing device may be described as one being used, but a person having ordinary skill in the art, the processing device may include a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that may include. For example, the processing device may include a plurality of processors or a processor and a controller. In addition, other processing configurations, such as parallel processors, are possible.

The software may include a computer program, code, instruction, or a combination of one or more of these, and configure the processing device to operate as desired, or process independently or collectively You can command the device. Software and/or data may be interpreted by a processing device or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. , Or may be permanently or temporarily embodied in the transmitted signal wave. The software may be distributed on networked computer systems, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, or the like alone or in combination. The program instructions recorded on the medium may be specially designed and constructed for the embodiments or may be known and usable by those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, and magnetic media such as floptical disks. -Hardware devices specifically configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language code that can be executed by a computer using an interpreter, etc., as well as machine language codes made by a compiler. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described by the limited drawings, those skilled in the art can apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and/or the components of the described system, structure, device, circuit, etc. are combined or combined in a different form from the described method, or other components Alternatively, even if replaced or substituted by equivalents, appropriate results can be achieved.

100: video classification system
110: image data generation unit
111: encoder
113: sampling unit
115: decoder
120: convolutional neural network
130: data classification unit

Claims (12)

An image data generator including an encoder that compresses input image data and extracts features, and a decoder that generates reconstructed image data that restores the input image data compressed by the encoder, and
Convolutional Neural Network
Including,
An image classification system comprising a data classification unit that classifies input image data using the encoder and the convolutional neural network unit. In claim 1,
An image classification system for training the data classification unit using the learning data generated by the image data generation unit. In claim 2,
When training the data classification unit, the image classification system to separate the encoder from the decoder to be placed in the front end of the composite neural network unit. In claim 3,
The image data generation unit is an automatic encoder image classification system. In claim 3,
The image data generation unit is a CCVAE (Conditional Convolutional Variational Autoencoder) image classification system. In claim 1,
The input image data is an image classification system which is image data obtained by photographing the surface of a product to be inspected by applying illumination to inspect for surface defects. Generating training data by an image data generator including an encoder that compresses input image data and extracts features, and a decoder that generates reconstructed image data that restores the input image data compressed by the encoder,
Constructing a data classification unit using the encoder and the convolutional neural network unit, and
Learning the data classification unit using the generated learning data
Control method of the video classification system comprising a. In claim 7,
When training the data classification unit, a control method of an image classification system that separates the encoder from the decoder so that it is disposed in front of the composite neural network unit. In claim 8,
The image data generation unit is an auto-encoder image classification system control method. In claim 8,
The image data generation unit CCVAE (Conditional Convolutional Variational Autoencoder) control method of the image classification system. In claim 7,
The input image data is a method of controlling an image classification system, which is image data obtained by photographing the surface of a product to be inspected by applying illumination to inspect for surface defects. In a computer-readable recording medium containing a program for performing a control method of the image classification system,
A set of instructions for generating learning data by an image data generation unit including an encoder for compressing input image data and extracting features, and a decoder for generating reconstructed image data by restoring the input image data compressed by the encoder,
A set of instructions for constructing a data classifier using a convolutional neural network that classifies the encoder and input image data according to extracted features, and
Instruction set for training the data classification unit using the generated training data
Computer-readable recording medium comprising a.

Images