KR20160144660A

KR20160144660A - Improved Classification Layer for Convolutional Neural Networks

Info

Publication number: KR20160144660A
Application number: KR1020150081084A
Authority: KR
Inventors: 김태경
Original assignee: 김태경
Priority date: 2015-06-09
Filing date: 2015-06-09
Publication date: 2016-12-19

Abstract

In order to use artificial neural networks to classify data, it is necessary to include a completely connected layer that classifies linearly inseparable data into linearly separable layers in the hidden layer of the neural network.
However, each neuron in the completely connected layer is connected to all neurons except for the bias neurons of the previous layer at different weights, so that no matter how similar the previous layers are, the neurons with different values can have different values and the data can be linearly separated It can be easily classified into other categories.
To solve this problem, the convolution neural network uses the convolution layer and the subsampling layer to downsample the image, and then connects the layer with the sufficiently reduced size to the classification layer to reduce the influence of the classification layer on the image deformation .
However, a fully connected layer is very vulnerable to image deformation, which can lead to misclassification even if the layer is sufficiently reduced in size.
In order to solve this problem, the present invention proposes a method of horizontally connecting the convolution layer to the existing classification layer, which is a fully connected layer, by improving the classification layer to be resistant to deformation of the image. Similar images can be partially matched, although not entirely consistent, by connecting the convolution layer, which is partially connected to the previous layer, so that some neurons in this layer can have the same value from the partially identical image, It is possible to classify images closer to similar images in space to improve the recognition performance of the images.
The accuracy of the MNIST handwritten digit and CIFAR-10 problems was improved by 0.11% and 2.86%, respectively, by replacing the existing fully connected layer with a horizontal convolution layer.

Description

[0002] Improved Classification Layer for Convolutional Neural Networks [

The present invention relates to a method for improving the performance of a convolutional neural network, and more particularly to a method for improving the performance of a convolutional neural network by improving the classification layer, which is one of the hidden layers of the convolutional neural network.

Neural networks can be used to classify data.

The above diagram is a geometric representation of the process of classifying nine data into three categories with a multilayer perceptron, a type of neural network. Each neuron represents one coordinate of the data dimension. The fully connected layer, which is the middle layer, classifies the data in the previous layer as linearly separable, and the classified data is categorized by the decision boundary of the output layer.

Each neuron in the fully connected layer is connected to all neurons except for the bias neurons in the previous layer at different weights so that no matter how similar the previous layers are, Lt; / RTI >

Images are categorized by what they contain. For numeric images, even if the position or color of a number in an image changes, the category of the image does not change. However, when neural networks are used for image classification, neurons in completely connected layers may have different values for slight movement or deformation of the image, which causes data in the coordinate space to be easily shifted to other categories of regions, thereby degrading classification accuracy .

To solve this problem, the convolution neural network downsamples the image by connecting the convolution layer and the sub-sampling layer between the input layer and the fully connected layer (splitting layer). By connecting the layer with a sufficiently reduced size to the classification layer, it reduces the influence of the classification layer on the image deformation and further improves the classification accuracy.

However, even if the effects of image deformation are sufficiently reduced by the next sampling process, the classification layer is still very vulnerable to deformation and can lead to misclassification.

1. Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, " Gradient-based learning applied to document, "Proceedings of the IEEE, vol.86, no.11, pp. 2278-2324, 1998. 2. D. Ciresan, U. Meier and J. Schmidhuber, " Multi-column deep neural networks for image classification, "In Computer Vision and Pattern Recognition, 3642-3649, June. 3. P. Y. Simard, D. Steinkraus, and J. C. Platt, "Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis," In 2013 12th International Conference on Document Analysis and Recognition, vol. 2, pp. 958-958, August 2003.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems and to improve the classification accuracy of the convolutional neural network by improving the classification layer which is very vulnerable to image deformation to be robust against deformation.

The present invention relates to an improved classification layer for convolutional neural networks for this purpose which comprises a convolution layer having a characteristic which is robust to the deformation of the image and which comprises a conventional fully connected layer for maintaining the role of classifying the data as linearly separable, Is horizontally connected to the classification layer.

The effect of the present invention is that the classification layer can perform the existing role of classifying the data to be linearly separable while at the same time being robust against the deformation of the image. The convolution layer is characterized by being partially connected to the previous layer, where the value of each neuron is determined by a portion of the previous layer, unlike a completely connected layer. Similar images may not be globally consistent, but may be partially consistent. Some of the neurons of the convolution layer horizontally connected to the classification layer have the same value from the partially matched image and when the output layer generates the decision boundary by transmitting such partial information to the output layer, . That is, some neurons in the convolution layer may have the same value from similar images, while all neurons in the totally connected layer have very different values from similar images, so that the classification layer positions data on the coordinate space more closely to similar images, The classification accuracy can be improved.

When classifying the MNIST handwritten digit and CIFAR-10 problems after applying the improved classification layer to the convolution neural network, the error of 0.96% and 36.04% using the conventional classification layer could be reduced to 0.85% and 33.18%, respectively .

1. Drawing 1 is an explanatory diagram of a convolution neural network
2. FIG. 2 is an explanatory diagram of the convolution layer
2. FIG. 3 is an explanatory diagram

1 is an explanatory diagram of a convolutional neural network.

1, the concealment layer of the convolutional neural network consists of convolution layers 20 and 40, sub-sampling layers 30 and 50, and a grading layer 60.

2 is an explanatory diagram of a convolution layer used in a convolutional neural network.

2, each neuron of the convolution layer 80 (one of which represents a neuron) is connected to a portion of the neuron of the previous layer 90,

Is calculated as follows.

here

The number of

feature maps

81, 82, 83 of the previous layer,

Of the previous floor

The neuron in the second feature map,

Is the weight that connects the neurons of the previous layer to the convolution layer,

Lt; RTI ID = 0.0 >

The bias neurons in the second feature map and

Is an active function. The active function used here is a hyperbolic tangent function.

Neurons in a feature map share a weight.

The sub-sampling method used here is max-pooling, in which each neuron in the subsampling layer is connected so that it does not overlap with 2x2 neurons in the previous layer, taking the largest value among connected neurons.

here

Is a sub-sampling layer neuron. The feature maps of the subsampling layers correspond one-to-one with feature maps of the convolution layer (20, 30).

3 is an explanatory diagram of an improved classification layer.

Referring to FIG. 3, the layer 111 and the convolution layers 112 and 113, which are completely connected to the classification layer 110, are horizontally connected. At this time, two or more convolution layers having feature maps of different sizes may be connected. For example, if two convolution layers are connected when the classification layer is connected to a previous layer 100 of 5x5 size, the classification layer is connected to the fully connected layer, 4x4 neurons of the previous layer, And a convolution layer having a 3x3 feature map coupled with 3x3 neurons of the previous layer.

Description of the Related Art
10: input layer 20: convolution layer 30: sub-sampling layer
40: convolution layer 50: subsampling layer 60:
70: Output layer
80: convolution layer 81-83: feature map 90: previous layer
100: previous layer 110: improved classification layer 120: output layer
111: Fully connected layer
112: Convolution layer (feature map size: 2x2)
113: Convolution layer (feature map size: 3x3)

Claims

Characterized in that a fully connected layer and a convolution layer are horizontally connected which can be used as a concealing layer of an artificial neural network.

2. The layer of claim 1, wherein the convolution layer can be comprised of two or more convolution layers having feature maps of one or different sizes.