KR20190078710A - Image classfication system and mehtod - Google Patents

Abstract

According to an embodiment of the present invention, provided is an image classification system, which comprises: a single class model classifying whether an input image is a positive class which is an object of interest; and a support model generating counterfeit class data which is negative class data which is an object of non-interest and not over fitted with positive class data belonging to the positive class. The single class model performs classification learning on the positive class data with the counterfeit class data.

Description

IMAGE CLASSIFICATION SYSTEM AND MEHTOD [0002]

The present disclosure relates to an image classification system and method.

In the age of Big Data, which collects vast amounts of data at high speed, much research has been done to solve large and small problems using machine learning technology. Among them, the technique of predicting the category to which the element belongs is called classification. The classification is divided into a multiclass classification and a binary classification according to the number of categories.

The multi-class classification includes three or more categories, and the binary classification includes only positive and negative in two categories. Generally, each of the positive class and negative class data is required for general binary classification. However, depending on the situation, only positive class data can be obtained as a training set. In the context of a binary class, predicting the category to which an unknown data item belongs, using only positive class data as a training set, is referred to as a one-class classification.

Single classification techniques are useful for developing personalized image retrieval systems. For example, you can predict which images users will be interested in and recommend them to users. In this situation, we can collect a number of logs of user activity that contain images of interest to the user. Considering the image that the user is interested in as positive class data and considering the image that the user is not interested in as negative class data, prediction of whether or not the user is interested in a particular image can be implemented in binary class classification . Such binary classifications can be a single classification because they use only positive class data as training sets.

Several single classification studies for image retrieval systems are already underway, and safety and security issues, product quality, etc. can be predicted using a single classification scheme.

However, the existing single classification technique only learns the distribution of the positive class data and classifies the input data into one of the positive class data and the negative class data. In this case, there is a problem that the positive class data is over-fitted and the boundary thereof is excessively narrowed.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an image classification system and method that can solve the problem of overly narrowing the boundary of a positive class by over-fitting with a single class model for an image search system.

According to an aspect of the present invention, there is provided an image classification system including: a single class model for classifying whether an input image is a positive class of interest; and a class classifier for classifying falsification class data, which is negative class data that is not subject to positive class data belonging to the positive class, And the single class model performs classification learning on the positive class and data and the falsified class data.

The support model may obtain classification results from the single class model and update the falsification class data through training based on the obtained classification results.

Wherein the classifying learning of the single class model and the updating operation based on the classification result of the supporting model are repeated up to a predetermined k to fox epochs so that the falsification class data does not coincide with the distribution of the positive class samples, May be a negative data sample.

 The single class model may perform classification learning using a validity verification set including the falsification class data and positive class data generated during fox in k.

The single class model includes a convolution layer for applying a convolution kernel to an input, a first pooling layer for applying a Max-Pulling kernel to the volume-converted input through the convolution array, an average pooling kernel for the input And a fully-connected (FC) layer connected to all of the components of the input to which the average pulling kernel is applied to calculate each element to produce a result of the single class model.

The support model includes: a fully-connected layer (FC) layer connected to all elements of an input noise vector to generate an input of a predetermined volume by calculating each element; a convolution layer for applying a convolution kernel to an input; Lt; RTI ID = 0.0 > up-sampling < / RTI > kernel.

The falsification data may be generated by a generative adversarial net (GAN) framework.

According to another aspect of the present invention, there is provided an image classifying method comprising the steps of: classifying a single class model into fog class data and positive class data belonging to a positive class of interest; Generating fake class data that is negative class data of interest, and classifying whether the input image is the positive class of interest.

The step of generating the falsification class data may include the step of the support model obtaining the classification result in the classification learning step and the updating of the falsification class data through training based on the obtained classification result have.

An updating step based on the classifying learning step of the single class model and the classification result of the supporting model can be repeated up to the predetermined k to the epochs.

The step of performing classification learning may use a validity verification set including the falsification class data and positive class data generated during fox in k.

The generating of the counterfeit data may be performed by a generative adversarial net (GAN) framework.

The embodiment is a single class model for an image retrieval system. Using the GAN framework, the falsification class data generated in the support model can be used to reduce the generalization error, and the single class model can accurately learn the characteristics of the positive class.

FIG. 1A is a conceptual diagram showing an over sum condition in a conventional single classification technique. FIG.
FIG. 1B is a conceptual diagram showing a single classification result using random negative class data, and FIG. 1C is a conceptual diagram showing a single classification result using appropriate negative class data.
2 is a diagram illustrating an image classification system according to an embodiment.
3 is a diagram illustrating a network architecture of a single class model according to an embodiment.
4 is a diagram illustrating a network architecture of a support model according to an embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

In order to solve the technical problem, the image classification system according to the embodiment generates negative class data and uses it to create a discrimination function that does not excessively exceed the positive class data. As a result, it is possible to provide an image classification system using a new single classification scheme that uses falsified data generated by a generative adversarial net (GAN) framework as negative class data.

Existing single classification schemes have over - sum problems in positive class data.

FIG. 1A is a conceptual diagram showing an over sum condition in a conventional single classification technique. FIG.

The circular point represents the positive class data and the closed curve represents the crystal boundary. Since the learning machine only learns the distribution of the positive class data, the decision boundary is too close to the positive class data. Thus, generating appropriate negative class data helps to generate a discriminant function that is not overly positive class data. At this time, the negative class data should not be formed at random, but should be appropriately generated close to the positive class data.

FIG. 1B is a conceptual diagram showing a single classification result using random negative class data, and FIG. 1C is a conceptual diagram showing a single classification result using appropriate negative class data. 1B and 1C, the triangle point represents negative class data.

As shown in FIG. 1B, since the negative class data is randomly generated, the boundary determined as a result of learning is inappropriately too far from the positive class data.

As shown in Fig. 1C, in the case of learning using the appropriate negative class data, it is found that the negative class data is appropriately generated in the vicinity of the positive class data, and that the boundary determined as the learning result is not excessively wider than positive class data .

In the GAN framework, the generation model generates counterfeit data similar to positive class data as learning progresses. In the embodiment, in order to solve the problem of the existing single classification technique, the GAN framework can be used to generate negative class data that is not overly compliant with the positive class data. Hereinafter, 'negative class data not excessively included in positive class data' is referred to as falsified class data. The embodiment can generate falsification class data that is close to the positive class data and use it to learn the characteristics of the positive class data.

2 is a diagram illustrating an image classification system according to an embodiment.

As shown in FIG. 2, the image classification system 1 includes a support model 10 and a one-class classification model 20.

The support model 10 generates the falsification class data 3 from the input data 2 to support the learning of the single class model 20. The input data 2 may be a Gaussian noise vector. The single class model 20 can classify the positive class data 4 and the falsified class data 3 in single classification learning.

The support model 10 generates fake class data 3 for a validation set. Next, the single class model 20 performs a training to classify the falsification class data 3 and the positive class data 4. The support model 10 obtains the classification results from the single class model 20 and updates the falsification class data 3 through training based on the obtained classification results. The single class model 2 performs training to classify the updated fake class data 3 and the positive class data 4 from the support model 10.

As described above, the single class model 20 classifies the falsification class data 3 and the positive class data 4 generated in the support model 10 and outputs the falsification class data 3 3) is repeated up to k epoxs. Then, the falsification class data 3 may be a negative data sample which does not coincide with the distribution of the positive class samples but has a close distribution. The validation set thus generated may include fake class data 3 and positive class data 4 generated during the epochs in k.

The single class model 20 can be optimized by performing learning using a validation set. The optimized single class model 20 thus transforms the image input into the image classification system to classify it as a positive class.

First, the single class model 20 performs training to classify the falsification class data 3 and the positive class data 4 using a validation set. Next, the support model 10 obtains the classification result of the single class model 20 and generates the falsification class data 3. The above operation is repeated until F1 of the validation set has the highest value, and the single class model 20 when the F1 value is the highest is determined as the final single class model.

Through the learning process as described above, the single class model 20 learns the characteristics of the positive class data 4 appropriately. By using the fake class data 3 generated by the support model 10, the single class model 20 20 are optimized so as not to over sum the positive class data 4. Through the embodiment, the single class model after learning in the GAN framework classifies the positive class data and the actual negative class data very well.

3 is a diagram illustrating a network architecture of a single class model according to an embodiment.

As shown in FIG. 3, the input image according to the data input to the single class model 20 is 32 × 32 × 3. '32 × 32' means the width × height of the pixels constituting the image, and '3' means the RGB channel of the image. The volume of the input image is an example for explanation, and the invention is not limited thereto.

As shown in FIG. 3, the convolution kernel, which is a weight of the convolution layer according to the embodiment, is a 5 × 5 kernel, and LeakyReLu can be used as an activation function.

First, in the convolution array for the input image of the single class model 20, the convolution kernel 5,5 is applied to the input image to convert the input image into a volume of 32 x 32 x 64, In the layer, a max pooling kernel (2, 2) is applied. In the next convolutional ray, the convolution kernel 5,5 is applied to the input image of the volume of 32x32x64 so that the input image is converted into the volume of 16x16x128, and in the next pooling layer, The pooling kernel (2, 2) is applied. In the next convolutional ray, the convolution kernel 5,5 is applied to an input image having a volume of 16 × 16 × 128, so that the input image is converted into a volume of 8 × 8 × 256, and in the next pulling layer, The pooling kernel (2, 2) is applied. Finally, in the convolution array, the convolution kernel (5, 5) is applied to the input image of 8 × 8 × 256 volume so that the input image is converted into the volume of 4 × 4 × 1. In the pooling layer, (average pooling kernel) (5,5) is applied. At the FC (fully-connected) layer, it is connected to all 4 × 4 × 1 elements applied with an average pooling kernel, and each element is calculated to generate the embedding of a single class model.

4 is a diagram illustrating a network architecture of a support model according to an embodiment.

As shown in Fig. 4, the input data 2 input to the support model 10 may be a noise vector. At the FC layer, it is connected to all the elements of the noise vector, and each element is calculated to generate a 4 x 4 x 256 volume input. At the convolution layer, a convolution kernel (5,5) is applied to a 4 × 4 × 256 volume input to convert the input to a volume of 4 × 4 × 128. At the next layer, an upsampling kernel (2,2) is applied and at the convolution layer a convolution kernel (5,5) is applied to convert the input to a volume of 8x8x128. At the next layer, the upsampling kernel (2,2) is applied, and at the convolution layer, the convolution kernel (5,5) is applied to convert the input to a volume of 16 x 16 x 64. At the next layer, the upsampling kernel (2,2) is applied, and at the convolution layer, the convolution kernel (5,5) is applied and the input is converted to a volume of 32 x 32 x 3 and output. This is the same volume as the input image of the single class model 20. The volume of the output of the support model 10 can be determined according to the input image volume.

3 and 4, the network architecture of the support model 10 and the single class model 20 according to the embodiment has been described, but the invention is not limited thereto. The convolution kernel, the max pooling kernel, the upsampling kernel, etc. can be modified, and the layers constituting the network architecture can be modified.

CIFAR-10 and CIFAR-100 can be applied to test classification results through the examples. For reference, the CIFAR-10 consists of 10 classes of 32 × 32 color images. The test for the example used only five classes (airplane, car, bird, car, deer) 32 color image consisting of 32 x 32 color images. In the test for the example, only five classes (beaver, dolphin, otter, seal, whale) were used. The video data of each class is regarded as positive class data. In the test for the embodiment, negative class data can be randomly extracted from nine classes except the positive class selected in CIFAR-10 and CIFAR-100.

For example, in CIFAR-10, the training set of the single class model (20) uses 4,500 positive class data, uses 1,000 positive class data and 1,000 negative class data in the test set, 500 positive class data and 500 fake class data are used.

In the CIFAR-100, 450 positive class data is used as the training set of the single class model 20, 100 positive class data and 100 negative class data are used as the test set, and 50 positive classes Data and 50 fake class data are used.

First, the single class model 20 performs the process of generating fake class data for validation using the CIFAR-10 and CIFAR-100 training sets. The process may be repeated up to the appropriate number of foxes 10, 100, 200 to generate falsification class data for validation. Next, we use a training set and validation set of CIFAR-10, CIFAR-100 to perform a process to find an optimal single class model. The process of finding the optimal single-class model while checking for F1 can be repeated up to 100 in Fox. As a result, the final single-class model can be determined as the learned model when F1 of the validation set is the highest. F1 is a F1 score, which is a factor indicating the accuracy of the classification result in general terms, and can be calculated using the precision and the recall. F1 = 2 * (accuracy * recall) / (accuracy + recall)).

The classification performance of the GAN-based single class model according to the embodiment and the classification performance of the single class model based on the existing neural network can be compared.

Table 1 shows the test results for CIFAR-10 and the single-class model using fake class data generated through Fox 10 for validation.

Table 2 shows the test results for CIFAR-10 and single class model using fake class data generated through Fox for 100 for validation.

Table 3 is a table showing test results for CIFAR-10 and a single class model using fake class data generated through Fox for 200 for validation.

Table 4 shows the test result of the conventional single class model using CIFAR-10.

As shown in Tables 1 to 4, it can be seen that the F1 value according to the embodiment is higher than that of the conventional single class model. Particularly, as shown in Table 3, the test result of the single class model using the fake class data generated through Fox in CIFAR-10 and 200 for validity verification is 0.087 higher than the conventional test result.

Table 5 shows the test results for CIFAR-100 and single-class model using fake class data generated through Fox for 10 validation.

Table 6 shows the test results for CIFAR-100 and single-class model using fake class data generated through Fox for 100 for validation.

Table 7 is a table showing test results for CIFAR-100 and a single class model using fake class data generated through Fox for 200 for validation.

Table 8 shows the test result of the conventional single class model using CIFAR-100.

As shown in Tables 5 to 8, it can be seen that the F1 value according to the embodiment is higher than that of the conventional single class model. Especially, as shown in Table 3, the test result of the single class model using CIFAR-100 and fake class data generated through Fox for 100 for validation is 0.084 higher than the conventional one.

The embodiment is a single class model for an image retrieval system. Using the GAN framework, the falsification class data generated in the support model can be used to reduce the generalization error, and the single class model can accurately learn the characteristics of the positive class.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It belongs to the scope of right.

1: Image classification system
10: Support Model
20: Single class model

Claims (12)

A single class model that classifies the input image as a positive class of interest, and
And a support model for generating fake class data, which is negative class data that is not overly related to positive class data belonging to the positive class,
Wherein the single class model performs classification learning of the positive class and data and the falsified class data. The method according to claim 1,
Wherein the support model obtains a classification result from the single class model and updates the falsification class data through training based on the obtained classification result. 3. The method of claim 2,
The classification learning of the single class model and the updating operation based on the classification result of the supporting model are repeated up to the predetermined k to the epochs,
Wherein the falsification class data is a negative data sample that does not match the distribution of the positive class samples but has a close distribution. The method of claim 3,
The single-class model includes:
Wherein the classification learning is performed using the falsification class data generated during fox in k and the validity verification set including positive class data. The method according to claim 1,
The single-class model includes:
A convolution layer that applies a convolution kernel to the input,
A first pooling layer for applying a Max-Pulling kernel to the volume-converted input via the convolution array,
A second pooling layer for applying an average pooling kernel on the input, and
And a fully-connected (FC) layer coupled to all components of the input to which the average pooling kernel is applied to calculate each element to produce a result of a single class model. The method according to claim 1,
In the support model,
A fully-connected (FC) layer connected to all elements of the noise vector to be input, for calculating each element to generate a predetermined volume of input,
A convolution layer that applies a convolution kernel to the input, and
An image classification system that includes layers that apply an upsampling kernel to the input. The method according to claim 1,
Wherein the forged data is generated by a generative adversarial net (GAN) framework. The single class model performing learning to classify the falsification class data and the positive class data belonging to the positive class of interest;
Generating fake class data that is negative class data that is not an object of interest and that is not overly supported by the supporting model; And
Classifying the input image as the positive class of interest. 9. The method of claim 8,
Wherein the generating of the falsification class data comprises:
The support model acquiring the classification result in the classification learning step; And
And updating the falsification class data through training based on the obtained classification result. 10. The method of claim 9,
Wherein the classifying learning step of the single class model and the updating step based on the classification result of the supporting model are repeated up to a predetermined k to fox epochs. 11. The method of claim 10,
The step of performing the classification learning includes:
Wherein the classification learning is performed using the falsification class data generated during fox in k and the validity verification set including positive class data. 9. The method of claim 8,
Wherein the generating of the counterfeit data comprises:
A method of image classification performed by a generative adversarial net (GAN) framework.

Images