KR20230103163A - Modular neural network nasnet based feature extraction method and system for image classification - Google Patents

Abstract

A feature extraction method and system based on Modular Neural Network NASNet for image classification are disclosed. receiving an input image to a modular neural network including a plurality of modules configured according to the number of inputs and outputs of modules according to an embodiment; and extracting feature information for image classification of the input image through the modular neural network, wherein the modular neural network includes a plurality of neural network models composed of a plurality of modules connected in series to classify the image. It may be learned to extract feature information for.

Description

Modular Neural Network NASNet-based Feature Extraction Method and System for Image Classification

The description below relates to feature extraction techniques for image classification.

Modular Neural Networks consist of several neural network models connected together through a plurality of modules. These modular neural networks make managing and handling more basic neural network systems simpler. In this case, several neural networks work as modules, each solving part of the problem.

Modular neural networks use various neural networks to solve problems. Here, various neural networks play the role of modules to solve part of the problem. The topology of the network is that different nodes and modules in the network are connected to each other. The sequential topology of the overall architecture of the model consists of several hybrids connected in series.

A modular neural network NASNet-based feature extraction method and system for image classification can be provided.

A feature extraction method for image classification includes receiving an input image to a modular neural network including a plurality of modules configured according to the number of inputs and outputs of the modules; and extracting feature information for image classification of the input image through the modular neural network, wherein the modular neural network includes a plurality of neural network models composed of a plurality of modules connected in series to classify the image. It may be learned to extract feature information for.

The modular neural network may be constructed to forward propagate an input image, stop it at a pre-specified layer, and use the pre-trained neural network as a feature extractor to use the output of the layer as a feature.

The modular neural network consists of an encoder and a decoder, and a first module outputs a denoised image through image preprocessing on an image containing noise, and extracts feature information through transfer learning from a pretrained neural network. It may include a second module and a third module for classifying an image by using the extracted feature information as input data.

The second module may include an architecture of a neural network including a convolution block including a plurality of convolution layers, a maximum pooling layer, and a dropout layer.

The second module may learn feature information as the noise-removed image passes through a neural network including at least one convolution block and a dense layer.

The second module may learn a new training image through transfer learning while feature information is extracted through a neural network including a plurality of convolution blocks from which dense layers are removed.

The second module may remove a pooling layer from each of the plurality of convolution blocks from which the dense layer is removed, replace it with a maximum pooling layer, and extract feature information through the replaced maximum pooling layer.

The third module may train a machine learning model to acquire a classifier capable of recognizing a new image using feature information output through the second module as input data.

The third module, Stochastic Gradient Descent (SGD), Random Forest (RF), Support Vector Machine (SVM), Logistic Regression (LR), Decision Making A machine learning model may be trained through any one classification algorithm including Decision Trees (DT).

A feature extraction method for image classification may include a computer program stored in a non-transitory computer readable recording medium to execute the method in the feature extraction system.

The feature extraction system includes an image input unit for receiving an input image to a modular neural network including a plurality of modules configured according to the number of inputs and outputs of the modules; and a feature information extractor extracting feature information for image classification of the input image through the modular neural network, wherein the modular neural network includes a plurality of neural network models composed of a plurality of modules connected in series. Feature information for image classification may be learned to be extracted.

Modular neural networks can solve complex artificial intelligence problems, improve learning speed, and reduce the amount of feature extraction.

By using a modular neural network, the accuracy of image classification can be increased and the consumption of computer power can be reduced.

1 is a diagram for explaining the structure of a modular neural network in a feature extraction system according to an embodiment.
2 is a diagram for explaining a first module configured in a modular neural network according to an embodiment.
3 to 8 are diagrams for explaining a second module configured in a modular neural network according to an embodiment.
9 is a diagram for explaining a third module configured in a modular neural network according to an embodiment.
10 is a block diagram for explaining the configuration of a feature extraction system according to an exemplary embodiment.
11 is a flowchart illustrating a feature extraction method for image classification in a feature extraction system according to an exemplary embodiment.

Hereinafter, an embodiment will be described in detail with reference to the accompanying drawings.

1 is a diagram for explaining the structure of a modular neural network in a feature extraction system according to an embodiment.

The feature extraction system receives an input image to a modular neural network including a plurality of modules configured according to the number of inputs and outputs of the modules, and extracts feature information for image classification of the input image through the modular neural network. there is.

Such a modular neural network may be learned such that a plurality of neural network models composed of a plurality of modules are serially connected to extract feature information for image classification.

A module is a multilayer feedforward neural network defined as 3-tuples.

M = (a, b, H)

Here, a is the number of inputs of the module, b is the number of output nodes, and H includes the number of neurons in each hidden layer.

For example, a multilayer perceptron module M with 8 inputs, 12 neurons in the first hidden layer, 10 neurons in the second hidden layer, and 4 outputs can be described as M(8, 4, [12, 10]). there is. The internal structure of the neural network, such as the number of hidden layers and the number of neurons, can be selected independently of the overall architecture in each hidden layer.

In the embodiment, a modular neural network including three modules will be described as an example. In a modular neural network, a plurality of independent neural networks (neural networks) are simultaneously trained for a specific sub-task, and the results obtained through learning can be finally combined to perform a single task.

Modular Neural Networks (MNN): module 1+module 2+module 3

Module 1 (101) is a denoising autoencoder (DAE) that recognizes an image containing noise as input data and generates an image from which noise is removed as output data.

Module 2 (102) performs transfer learning using a pre-trained neural network for feature extraction. In extracting the feature information of each image, a pre-trained neural network (NASNet, NASNet-Large) can be used as a feature extractor.

Module 3 (103) is a classifier (new model) that uses the extracted feature information as input data for image classification.

Modular neural networks are simply a combination of technologies (two or more networks can be used as building blocks), efficiency, scalability, reduced model complexity, robustness and incrementality (combined networks grow incrementally). can

Feature extraction is a technique that reduces the feature amount of a dataset by generating new feature information from existing feature information. In the embodiment, modular neural network-based feature extraction using a pre-trained neural network (eg, NASNet) as a feature extractor to forward propagate an input image, stop at a pre-specified layer, and use the output of the layer as feature information. We will explain the approach. The classifier can be trained using the feature information extracted by this modular neural network-based feature extraction approach.

The proposed deep learning-based feature extraction approach for accurate image classification can be performed in three steps including (1) image preprocessing, (2) feature information extraction from denoised images, and (3) image classification.

에서 샘플링될 수 있다.

는 학습 예제로 사용될 수 있다. 노이즈 제거 오토인코더는 음의 로그 가능성에 대한 기울기 기반 근사 최소화로 학습할 수 있는 피드포워드 네트워크이다. Referring to FIG. 2, it is a diagram for explaining module 1. In module 1, images containing noise are analyzed and clean images are generated. Adding noise (e.g. Gaussian) to the input allows the model to learn features that matter to the data. In the case of a denoising autoencoder, noise is added to the input vector stochastically, partially corrupting the data. The model can be trained so that the undamaged original, i.e., uncorrupted data points, are predicted and output as output data. For example, an input may be sampled from a data set, and a corrupted version of the input may be mapped to a stochastic

can be sampled at

can be used as a learning example. A denoising autoencoder is a feedforward network that can be trained by minimizing a gradient-based approximation to the negative log likelihood.

Referring to FIGS. 3 to 8 , they are diagrams for explaining module 2. Feature Extraction helps extract the optimal shape. By selecting variables and binding them to type features, you can reduce the amount of data.

Referring to FIG. 3 , it relates to a neural network architecture design for image classification. In module 2, a convolution block 201 including a plurality of convolution layers, a max pooling layer, and a dropout layer may be configured. An image from which noise is removed in the input layer may be input as input data. As the noise-removed image passes through the convolution block and the dense layer, feature information may be learned. Accordingly, a class (eg, bicycle) of an image from which noise has been removed may be output as output data in the output layer.

Referring to FIG. 4, it relates to a general design of a neural network for image classification, and the three layers described in FIG. 3 may be grouped into one block (convolutional block). The convolution block 201 may be formed by a plurality of convolution layers, max pooling layers, and dropout layers.

Referring to FIG. 5 , it relates to a neural network architecture composed of a plurality of convolution blocks. You can receive an input image. In this case, the input image may be an image from which noise is removed. A neural network composed of multiple convolutional blocks is a convolutional network that can distinguish one aspect from another by assigning importance (learnable weights and biases) to various aspects/objects of an input image.

Referring to FIG. 6 , it relates to a neural network architecture in which a dense layer is removed from a neural network composed of a plurality of convolution blocks. Transfer learning can be performed while extracting feature information with NASNet. While feature information is extracted through a neural network including a plurality of convolution blocks from which dense layers have been removed, a new training image may be learned through transfer learning. The extracted feature information may be stored in a file.

NASNet stands for Neural Search Architecture (NAS) Network and is a machine learning model. NASNet, which frames the problem of finding the best CNN architecture as a reinforcement learning problem, was introduced. The evolution of CNN has gone through three stages: Deeper is Better, 　Architecture Engineering　and　AutoML. Each phase has a representative network architecture.

Transfer learning is the process of reusing a model built for a problem for another problem depending on some factors. Using transfer learning saves training time and resources and solves small data availability issues. NASNet as a feature extractor trains ImageNet data on pre-trained NASNet with fully connected layers removed.

Referring to FIG. 7 , it is a diagram illustrating a modular neural network. It is a modular neural network with a sequential topology composed of multiple modules connected in series. A plurality of layers may be serially connected to the modular neural network. The pre-trained feature extractor (module 2) 102 and the new model (module 3) 103 can be serially connected.

Referring to FIG. 8, it shows the use of the NASNet-Large model as the feature extractor 801. NASNet-based features can be extracted and dimensionality reduced.

NASNet-Large is a Convolutional Neural Network trained on over a million images from the ImageNet database. The network can classify images into 1000 object categories, such as keyboard, mouse, and pencil. As a result, the network can learn rich feature representations for a wide range of images. The size of the image input to the network is 331 x 331.

The advantage of using NASNet-Large is improved model accuracy and reduced computer power usage. Also, dimensionality reduction can be performed by a pre-trained model.

In NASNet, instead of removing the pooling layer and returning the max pooling layer, the output is used as the extracted feature. The pooling layer serves to reduce the spatial size of convoluted features. This is to reduce the computational power required to process data through dimensionality reduction. In addition, it is useful to maintain an effective training process of the model by extracting rotational and positional invariant domain features. There are two types of pooling including max pooling and average pooling. Max pooling returns the maximum value of the portion of the image covered by the kernel. On the other hand, average pooling returns the average of all values of the image part included in the kernel. When propagation is stopped in a pre-specified layer (eg, a dense layer or a pooling layer), a value is extracted from the specified layer, and the extracted value is treated as a feature vector.

In NASNet, if we stop propagation before the pooling layer, the last layer in the network will be the maximum pooling layer, and the output shape will be 7 x 7 x 512. Merging these volumes into feature vectors gives us a list of 7 x 7 x 512 = 25,088 values. The list of numbers serves as a feature vector used to quantify the input image.

9 is for explaining module 3 configured in a modular neural network according to an embodiment, and shows a new model using extracted feature information as input data for image classification. In detail, the feature information extracted in module 2 can be input to a new model (module 3). New feature information learned with respect to the feature information extracted through the new model may be obtained. Output data (eg, a bicycle) may be output through image classification of the obtained new feature information.

Module 3 can train a machine learning model to obtain a classifier capable of recognizing new images. With feature vectors, Stochastic Gradient Descent (SGD), Random Forest (RF), Support Vector Machine (SVM), Logistic Regression (LR), Decision Trees ( Machine learning algorithms such as Decision Trees (DT) can be trained.

10 is a block diagram for explaining the configuration of a feature extraction system according to an embodiment, and FIG. 11 is a flowchart for explaining a feature extraction method for image classification in the feature extraction system according to an embodiment.

The processor of the feature extraction system 100 may include an image input unit 1010 and a feature information extraction unit 1020 . Components of the processor may be representations of different functions performed by the processor according to control instructions provided by program codes stored in the feature extraction system. The processor and components of the processor may control the feature extraction system to perform steps 1110 to 1120 included in the feature extraction method for image classification of FIG. 11 . In this case, the processor and components of the processor may be implemented to execute instructions according to the code of an operating system included in the memory and the code of at least one program.

The processor may load a program code stored in a file of a program for a feature extraction method for image classification into a memory. For example, when a program is executed in the feature extraction system, the processor may control the feature extraction system to load a program code from a program file into a memory under the control of an operating system. At this time, each of the image input unit 1010 and the feature information extraction unit 1020 executes a command of a corresponding part of the program code loaded into the memory to perform the subsequent steps 1110 to 1120 with different functional expressions of the processor. can be picked up

In step 1110, the image input unit 1010 may receive an input image to a modular neural network including a plurality of modules configured according to the number of inputs and outputs of the modules. A denoising process may be performed on an input image input to the modular neural network. At this time, noise removal may be performed on the input image through one of the modules configured in the modular neural network that performs image preprocessing.

In operation 1120, the feature information extractor 1020 may extract feature information for image classification of the input image through a modular neural network. At this time, feature information is extracted from a pre-trained neural network through a module for extracting feature information among modules configured in the modular neural network, and features extracted through a module for classifying images among modules configured in the modular neural network. The information can be used again as input data to classify the image.

The devices described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA) , a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. The device can be commanded. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or provide instructions or data to a processing device. can be embodied in The software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program commands recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

As described above, although the embodiments have been described with limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (11)

In the feature extraction method for image classification,
receiving an input image to a modular neural network including a plurality of modules configured according to the number of inputs and outputs of the modules; and
extracting feature information for image classification of the input image through the modular neural network;
including,
The modular neural network,
A feature extraction method for image classification, characterized in that a plurality of neural network models composed of a plurality of modules are connected in series and learned to extract feature information for image classification. According to claim 1,
The modular neural network,
A feature extraction method for image classification, characterized in that it is built to forward propagate an input image, stop it in a pre-specified layer, and use a pre-trained neural network as a feature extractor to use the output of the layer as a feature. According to claim 2,
The modular neural network,
A first module composed of an encoder and a decoder that outputs a denoised image through image preprocessing on an image containing noise; a second module that extracts feature information from a pretrained neural network through transfer learning; A feature extraction method for image classification comprising a third module for classifying an image using feature information as input data. According to claim 3,
The second module,
A feature extraction method for image classification comprising a architecture of a neural network including a convolution block including a plurality of convolution layers, a maximum pooling layer, and a dropout layer. According to claim 4,
The second module,
A feature extraction method for image classification, characterized in that the feature information is learned as the noise-removed image passes through a neural network including at least one convolution block and a dense layer. According to claim 4,
The second module,
A feature extraction method for image classification, characterized in that a new training image is learned through transfer learning while feature information is extracted through a neural network including a plurality of convolution blocks from which dense layers have been removed. According to claim 6,
The second module,
A feature extraction method for image classification, characterized in that the pooling layer is removed from each of the plurality of convolution blocks from which the dense layer is removed, replaced with a maximum pooling layer, and feature information is extracted through the replaced maximum pooling layer. . According to claim 2,
The third module,
A feature extraction method for image classification, characterized in that for learning a machine learning model to obtain a classifier capable of recognizing a new image using the feature information output through the second module as input data. According to claim 8,
The third module,
Stochastic Gradient Descent (SGD), Random Forest (RF), Support Vector Machine (SVM), Logistic Regression (LR), Decision Trees (DT) ) A feature extraction method for image classification, characterized in that for learning a machine learning model through any one classification algorithm containing. A computer program stored in a non-transitory computer readable recording medium to execute the feature extraction method for image classification according to any one of claims 1 to 9 in the feature extraction system. In the feature extraction system,
an image input unit for receiving an input image to a modular neural network including a plurality of modules configured according to the number of inputs and outputs of the modules; and
A feature information extraction unit extracting feature information for image classification of the input image through the modular neural network.
including,
The modular neural network,
A feature extraction system, characterized in that a plurality of neural network models composed of a plurality of modules are connected in series and learned to extract feature information for image classification.

Images