KR20160144660A - Improved Classification Layer for Convolutional Neural Networks - Google Patents
Improved Classification Layer for Convolutional Neural Networks Download PDFInfo
- Publication number
- KR20160144660A KR20160144660A KR1020150081084A KR20150081084A KR20160144660A KR 20160144660 A KR20160144660 A KR 20160144660A KR 1020150081084 A KR1020150081084 A KR 1020150081084A KR 20150081084 A KR20150081084 A KR 20150081084A KR 20160144660 A KR20160144660 A KR 20160144660A
- Authority
- KR
- South Korea
- Prior art keywords
- layer
- convolution
- neurons
- classification
- image
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G06K9/00536—
-
- G06K9/62—
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
In order to use artificial neural networks to classify data, it is necessary to include a completely connected layer that classifies linearly inseparable data into linearly separable layers in the hidden layer of the neural network.
However, each neuron in the completely connected layer is connected to all neurons except for the bias neurons of the previous layer at different weights, so that no matter how similar the previous layers are, the neurons with different values can have different values and the data can be linearly separated It can be easily classified into other categories.
To solve this problem, the convolution neural network uses the convolution layer and the subsampling layer to downsample the image, and then connects the layer with the sufficiently reduced size to the classification layer to reduce the influence of the classification layer on the image deformation .
However, a fully connected layer is very vulnerable to image deformation, which can lead to misclassification even if the layer is sufficiently reduced in size.
In order to solve this problem, the present invention proposes a method of horizontally connecting the convolution layer to the existing classification layer, which is a fully connected layer, by improving the classification layer to be resistant to deformation of the image. Similar images can be partially matched, although not entirely consistent, by connecting the convolution layer, which is partially connected to the previous layer, so that some neurons in this layer can have the same value from the partially identical image, It is possible to classify images closer to similar images in space to improve the recognition performance of the images.
The accuracy of the MNIST handwritten digit and CIFAR-10 problems was improved by 0.11% and 2.86%, respectively, by replacing the existing fully connected layer with a horizontal convolution layer.
Description
The present invention relates to a method for improving the performance of a convolutional neural network, and more particularly to a method for improving the performance of a convolutional neural network by improving the classification layer, which is one of the hidden layers of the convolutional neural network.
Neural networks can be used to classify data.
The above diagram is a geometric representation of the process of classifying nine data into three categories with a multilayer perceptron, a type of neural network. Each neuron represents one coordinate of the data dimension. The fully connected layer, which is the middle layer, classifies the data in the previous layer as linearly separable, and the classified data is categorized by the decision boundary of the output layer.
Each neuron in the fully connected layer is connected to all neurons except for the bias neurons in the previous layer at different weights so that no matter how similar the previous layers are, Lt; / RTI >
Images are categorized by what they contain. For numeric images, even if the position or color of a number in an image changes, the category of the image does not change. However, when neural networks are used for image classification, neurons in completely connected layers may have different values for slight movement or deformation of the image, which causes data in the coordinate space to be easily shifted to other categories of regions, thereby degrading classification accuracy .
To solve this problem, the convolution neural network downsamples the image by connecting the convolution layer and the sub-sampling layer between the input layer and the fully connected layer (splitting layer). By connecting the layer with a sufficiently reduced size to the classification layer, it reduces the influence of the classification layer on the image deformation and further improves the classification accuracy.
However, even if the effects of image deformation are sufficiently reduced by the next sampling process, the classification layer is still very vulnerable to deformation and can lead to misclassification.
SUMMARY OF THE INVENTION The present invention has been made to solve the above problems and to improve the classification accuracy of the convolutional neural network by improving the classification layer which is very vulnerable to image deformation to be robust against deformation.
The present invention relates to an improved classification layer for convolutional neural networks for this purpose which comprises a convolution layer having a characteristic which is robust to the deformation of the image and which comprises a conventional fully connected layer for maintaining the role of classifying the data as linearly separable, Is horizontally connected to the classification layer.
The effect of the present invention is that the classification layer can perform the existing role of classifying the data to be linearly separable while at the same time being robust against the deformation of the image. The convolution layer is characterized by being partially connected to the previous layer, where the value of each neuron is determined by a portion of the previous layer, unlike a completely connected layer. Similar images may not be globally consistent, but may be partially consistent. Some of the neurons of the convolution layer horizontally connected to the classification layer have the same value from the partially matched image and when the output layer generates the decision boundary by transmitting such partial information to the output layer, . That is, some neurons in the convolution layer may have the same value from similar images, while all neurons in the totally connected layer have very different values from similar images, so that the classification layer positions data on the coordinate space more closely to similar images, The classification accuracy can be improved.
When classifying the MNIST handwritten digit and CIFAR-10 problems after applying the improved classification layer to the convolution neural network, the error of 0.96% and 36.04% using the conventional classification layer could be reduced to 0.85% and 33.18%, respectively .
1.
2. FIG. 2 is an explanatory diagram of the convolution layer
2. FIG. 3 is an explanatory diagram
1 is an explanatory diagram of a convolutional neural network.
1, the concealment layer of the convolutional neural network consists of
2 is an explanatory diagram of a convolution layer used in a convolutional neural network.
2, each neuron of the convolution layer 80 (one of which represents a neuron) is connected to a portion of the neuron of the
here
The number of
Neurons in a feature map share a weight.
The sub-sampling method used here is max-pooling, in which each neuron in the subsampling layer is connected so that it does not overlap with 2x2 neurons in the previous layer, taking the largest value among connected neurons.
here
Is a sub-sampling layer neuron. The feature maps of the subsampling layers correspond one-to-one with feature maps of the convolution layer (20, 30).3 is an explanatory diagram of an improved classification layer.
Referring to FIG. 3, the
Description of the Related Art
10: input layer 20: convolution layer 30: sub-sampling layer
40: convolution layer 50: subsampling layer 60:
70: Output layer
80: convolution layer 81-83: feature map 90: previous layer
100: previous layer 110: improved classification layer 120: output layer
111: Fully connected layer
112: Convolution layer (feature map size: 2x2)
113: Convolution layer (feature map size: 3x3)
Claims (2)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020150081084A KR20160144660A (en) | 2015-06-09 | 2015-06-09 | Improved Classification Layer for Convolutional Neural Networks |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020150081084A KR20160144660A (en) | 2015-06-09 | 2015-06-09 | Improved Classification Layer for Convolutional Neural Networks |
Publications (1)
Publication Number | Publication Date |
---|---|
KR20160144660A true KR20160144660A (en) | 2016-12-19 |
Family
ID=57735346
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
KR1020150081084A KR20160144660A (en) | 2015-06-09 | 2015-06-09 | Improved Classification Layer for Convolutional Neural Networks |
Country Status (1)
Country | Link |
---|---|
KR (1) | KR20160144660A (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE202017005181U1 (en) | 2016-11-01 | 2017-10-16 | Hyundai Mobis Co., Ltd. | In-Wheel-working device |
KR101874968B1 (en) * | 2017-07-03 | 2018-08-02 | (주)시정 | Visibility measuring system base on image information and method for using the same |
KR20190039459A (en) * | 2017-10-04 | 2019-04-12 | 주식회사 스트라드비젼 | Learning method and learning device for improving performance of cnn by using feature upsampling networks, and testing method and testing device using the same |
KR20190039458A (en) * | 2017-10-04 | 2019-04-12 | 주식회사 스트라드비젼 | Method and device for generating feature maps by using feature upsampling networks |
KR20210152249A (en) * | 2020-06-08 | 2021-12-15 | 고려대학교 산학협력단 | Apparatus and method for artificial intelligence based automatic analysis of video fluoroscopic swallowing study |
-
2015
- 2015-06-09 KR KR1020150081084A patent/KR20160144660A/en unknown
Non-Patent Citations (3)
Title |
---|
1. Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, "Gradient-based learning applied to document," Proceedings of the IEEE, vol.86, no.11, pp.2278-2324, 1998. |
2. D. Ciresan, U. Meier and J. Schmidhuber, "Multi-column deep neural networks for image classification," In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on pp. 3642-3649, June.2012. |
3. P. Y. Simard, D. Steinkraus, and J. C. Platt, "Best practices for convolutional neural networks applied to visual document analysis," In 2013 12th International Conference on Document Analysis and Recognition, vol. 2, pp. 958-958, August.2003. |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE202017005181U1 (en) | 2016-11-01 | 2017-10-16 | Hyundai Mobis Co., Ltd. | In-Wheel-working device |
KR101874968B1 (en) * | 2017-07-03 | 2018-08-02 | (주)시정 | Visibility measuring system base on image information and method for using the same |
KR20190039459A (en) * | 2017-10-04 | 2019-04-12 | 주식회사 스트라드비젼 | Learning method and learning device for improving performance of cnn by using feature upsampling networks, and testing method and testing device using the same |
KR20190039458A (en) * | 2017-10-04 | 2019-04-12 | 주식회사 스트라드비젼 | Method and device for generating feature maps by using feature upsampling networks |
KR20210152249A (en) * | 2020-06-08 | 2021-12-15 | 고려대학교 산학협력단 | Apparatus and method for artificial intelligence based automatic analysis of video fluoroscopic swallowing study |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
KR102516360B1 (en) | A method and apparatus for detecting a target | |
Rahman et al. | A new benchmark on american sign language recognition using convolutional neural network | |
KR102030628B1 (en) | Recognizing method and system of vehicle license plate based convolutional neural network | |
Pinto Jr et al. | Static hand gesture recognition based on convolutional neural networks | |
CN109584248B (en) | Infrared target instance segmentation method based on feature fusion and dense connection network | |
Pastor-Pellicer et al. | Insights on the use of convolutional neural networks for document image binarization | |
KR20160144660A (en) | Improved Classification Layer for Convolutional Neural Networks | |
CN108921119B (en) | Real-time obstacle detection and classification method | |
WO2018052587A1 (en) | Method and system for cell image segmentation using multi-stage convolutional neural networks | |
Iniyan et al. | Plant disease identification and detection using support vector machines and artificial neural networks | |
Chu et al. | Manga facenet: Face detection in manga based on deep neural network | |
CN111950649A (en) | Attention mechanism and capsule network-based low-illumination image classification method | |
CN110109060A (en) | A kind of radar emitter signal method for separating and system based on deep learning network | |
CN111191583A (en) | Space target identification system and method based on convolutional neural network | |
US9224207B2 (en) | Segmentation co-clustering | |
Burie et al. | ICFHR2016 competition on the analysis of handwritten text in images of balinese palm leaf manuscripts | |
CN109858467B (en) | Face recognition method and device based on key point region feature fusion | |
Medina et al. | Comparison of CNN and MLP classifiers for algae detection in underwater pipelines | |
CN107491729B (en) | Handwritten digit recognition method based on cosine similarity activated convolutional neural network | |
CN111582033A (en) | Garbage classification identification method and system and computer readable storage medium | |
Pandiya et al. | Analysis of deep learning architectures for object detection-a critical review | |
Zhou et al. | Tree species identification based on convolutional neural networks | |
CN109002771A (en) | A kind of Classifying Method in Remote Sensing Image based on recurrent neural network | |
Wu | CNN-Based Recognition of Handwritten Digits in MNIST Database | |
Anggraeny et al. | Image preprocessing analysis in handwritten Javanese character recognition |