KR20190118387A - Convolutional neural network based image processing system and method - Google Patents

Abstract

The present invention provides an image processing system based on a convolutional neural network and a method thereof which can improve image classification accuracy. According to an embodiment of the present invention, the image processing system based on a convolutional neural network comprises: a convolution layer unit to use a plurality of kernel filters to extract a feature value of an object from input data; an activation layer unit to perform a conversion task on the feature value extracted by the convolution layer unit by using a nonlinear activation function; a polling layer unit to use maximum polling calculation on an output value of the activation layer unit to reduce dimensions and remove or suppress noise; a classification output layer unit to output a classification prediction value for the input data by forward calculation using the output value of the polling layer unit; a loss calculation layer unit to compare the classification prediction value and a predetermined target value to calculate a loss value corresponding to an error value; a reverse calculation unit to calculate a partial differentiation value for the loss value by reverse calculation to acquire a correction value for a parameter of each layer unit; and a classification learning unit to perform an update on the parameter by a gradient descent method using a learning rate extracted by the correction value and a prescribed quantity of learning data.

Description

Convolutional neural network based image processing system and method {CONVOLUTIONAL NEURAL NETWORK BASED IMAGE PROCESSING SYSTEM AND METHOD}

Embodiments of the present invention relate to an image processing technique, and more particularly, to a system and method that can improve image classification accuracy by processing an image using a composite product neural network technique.

Recognition is a key goal of computer vision. There are many standard algorithms and objective measurement algorithms for objectively comparing computer vision recognition methodologies and recognition systems. Recognition is divided into instance recognition and category recognition according to the object. Case recognition is to recognize a specific object. In the past, recognition algorithms often face problems such as lighting changes and geometric transformations in real-world environments. To solve this problem, it is necessary to develop an image processing algorithm to improve practicality. Case recognition makes class modeling easy, while category recognition makes it difficult to model classes because of the variety of objects in the category. The problem of category recognition is that there is an intra-class variation problem. To solve this problem, not only computer vision algorithms but also patterns recognition and machine learning are required.

The existing case recognition or category recognition developer proceeds with the process of feature extraction and matching in the algorithm. The feature extraction step uses feature extraction algorithms such as edge detection (LOG, Canny, edge segment), local feature detection (SSD, WSSD, SIFT, SURF), feature description (SIFT, PCA-SIFT). The matching step is an object recognition method using algorithms and features such as neighbor search (kd tree, KNN, hashing) and transform estimation (M-estimator, RANSAC).

Since feature extraction algorithms such as SIFT / SURF are invariant to movement and rotation, they are widely used among image recognition methods. In recent years, the convolutional neural network (CNN) technology of image deep learning, which is an image deep learning method that utilizes the advantages of the feature extraction algorithm and the high reliability of matching, is noted. Is getting.

In the present invention, a simple method of processing feature extraction and matching independently from each other in a process which is a disadvantage of the recognition process of an existing image recognition system, and using an image processing system integrated with feature extraction and matching using a composite product neural network technology Design and implementation. To this end, the existing image recognition system can be operated and analyzed using a composite product neural network, and the system is designed and implemented to automatically process the feature extraction and matching process for images.

Related prior arts include Republic of Korea Patent Publication No. 10-1834791 (name of the invention: a method for providing classification information of main capital and woodblock using a synthetic product neural network, registration date: February 27, 2018).

One embodiment of the present invention provides a multiplicative neural network based image processing system and method that can improve the image classification accuracy by processing the image using the multiplicative neural network technology.

The problem to be solved by the present invention is not limited to the problem (s) mentioned above, and other object (s) not mentioned will be clearly understood by those skilled in the art from the following description.

Synthetic product neural network based image processing system according to an embodiment of the present invention comprises a composite product layer unit for extracting a feature value of the object from the input data using a plurality of kernel filters; An activation layer unit which performs a transform operation on a feature value extracted by the composite product layer unit using a nonlinear activation function; A pooling layer unit which reduces a dimension, removes and suppresses noise by using a maximum polling operation on an output value of the activation layer unit; A classification output layer unit for outputting a classification prediction value for the input data through an omnidirectional operation using the output value of the pooling layer unit; A loss calculating layer unit comparing the classification prediction value with a predetermined target value and calculating a loss value corresponding to the error value; A reverse calculation unit calculating a partial differential value of the loss value through a reverse operation to obtain a correction value for a parameter of each layer unit; And a classification learning unit for updating the parameter through a gradient descent method using the correction value and the learning rate derived through a certain amount of learning data.

The activation layer may perform a conversion operation on the feature value by using a reLU activation function.

The reverse calculation section calculates a partial differential value for the first parameter of the classification output hierarchy in the direction of the classification output hierarchy in the loss calculation hierarchy, and a second derivative in the direction of the last hidden hierarchy in the classification output hierarchy. Partial derivative values for parameters are calculated, and partial derivative values for third parameters between hidden layers are calculated to obtain correction values for the parameters (the first to third parameters) of each layer part. can do.

In the convolutional neural network based image processing system according to an exemplary embodiment of the present invention, a convolution calculation is performed in parallel using a nonlinear activation function on two or more feature values output by the convolutional layer, and the parallel A new feature that combines the characteristics of the input data by performing a matching operation on the concatenation calculation result to obtain a multidimensional result value, and performing a dimensional transform operation on the multidimensional result value by using a kernel filter and a linear activation function. The apparatus may further include an inception layer unit that extracts a value.

The composite product neural network based image processing system according to an embodiment of the present invention generates multidimensional data through the inception layer unit and the composite product layer unit with respect to the input data, and matches the input to match the number of dimensions of the multidimensional data. The apparatus may further include a skip connection layer unit for converting the number of dimensions of data, combining the multidimensional data and the input data having the converted number of dimensions, and suppressing a problem of feature loss of the input data by using a linear activation function.

The composite product neural network based image processing system according to an embodiment of the present invention designs an He initial value algorithm based on a sample standard deviation calculated using the quantity of parameters of each layer unit, and is based on the He initial value algorithm. The apparatus may further include a parameter initial value design unit configured to initialize the parameter.

The composite product neural network based image processing system according to an embodiment of the present invention is implemented in the form of a mini batch for each of the input data, calculates an average value of the mini batch, and uses the average value of the mini batch. And a mini-batch normalization design unit for calculating a variance value of the mini-batch, and obtaining each value of the normalized mini-batch using the mean value and the variance value of the mini-batch for each input value of the mini-batch, and the composite product The hierarchical unit may use each value of the normalized mini batch as the input data.

The composite product neural network based image processing system according to an embodiment of the present invention calculates the momentum of the parameter, calculates the correction value of the learning rate using the derivative value of the parameter, and corrects the momentum and the learning rate. The method may further include a gradient descent optimization design unit that performs a correction operation of the parameter using a value.

Synthetic product neural network based image processing method according to an embodiment of the present invention comprises the steps of extracting the feature value of the object from the input data using a plurality of kernel filter of the composite product layer of the image processing system; Performing, by the activation layer unit of the image processing system, a transform operation on the feature value extracted by the composite product layer unit using a nonlinear activation function; A pooling layer unit of the image processing system reduces the dimension, removes and suppresses the noise by using a maximum polling operation on an output value of the activation layer unit; Outputting, by the classification output layer unit of the image processing system, a classification prediction value for the input data through an omnidirectional operation using the output value of the pooling layer unit; A loss calculating layer unit of the image processing system comparing the classification prediction value with a predetermined target value and calculating a loss value corresponding to the error value; A backward calculation unit of the image processing system calculates a partial differential value of the loss value through a backward operation to obtain a correction value for a parameter of each layer unit; And performing, by the classification learning unit of the image processing system, updating the parameter through a gradient descent method using the correction value and the learning rate derived through a predetermined amount of learning data.

Synthetic product neural network based image processing method according to an embodiment of the present invention includes the step of calculating the momentum of the parameter by the gradient descent optimization design of the image processing system; Calculating, by the gradient descent optimization design unit, a correction value of the learning rate using the derivative value of the parameter; And performing a correction operation of the parameter by using the gradient descent optimization design unit by using the correction value of the momentum and the learning rate.

Specific details of other embodiments are included in the detailed description and the accompanying drawings.

According to an embodiment of the present invention, image classification accuracy may be improved by processing an image by using a composite product neural network technology.

According to an embodiment of the present invention, multi-dimensional image data can be processed in parallel, and a multiplication / pooling operation can be performed on appropriate multi-dimensional image data without burden of memory.

According to an embodiment of the present invention, by performing a skip process through the skip connection layer, it is possible to suppress the problem of feature loss of the training data according to the dip of the learning model.

According to one embodiment of the present invention, by performing the parameter initialization operation based on the He initial value algorithm, it is possible to suppress the problem of expression power limitation and parameter loss.

According to an embodiment of the present invention, since the data mini-batch normalization algorithm is used, not only the over fitting problem can be suppressed but also the fast learning can be performed.

According to an embodiment of the present invention, by performing the optimization design using the advanced (improved) gradient descent method based on the Adam algorithm, it is possible to perform the acceleration learning and to calculate the optimal parameter in the time limit.

1 is a block diagram illustrating a composite product neural network based image processing system according to an exemplary embodiment of the present invention.
2 is a diagram illustrating a basic structure of a proposed learning model.
3 is a diagram illustrating a reinforcement structure of the proposed learning model.
4 is a diagram illustrating a complex structure of a proposed learning model.
5 shows a 2x2 size result processed using a 3x3 size filter algorithm on input 4x4 size data.
6 is a diagram illustrating a detailed calculation process of the filter operation.
7 is a diagram illustrating a padding process.
8 illustrates a process of reducing the size of data using a 2 × 2 maximum value pooling operation.
9 illustrates a process of a multiplication / pooling framework processing a 3D image.
FIG. 10 illustrates a structure of the inception layer unit of FIG. 1.
FIG. 11 illustrates a structure of a skipped connection layer unit of FIG. 1.
12 is a diagram illustrating a backward calculation algorithm of a learning model.
13 to 15 illustrate a detailed backward calculation process.
It is a figure which shows the process of the gradient descent method.
17 is a flowchart of a process of learning using mini-batch normalization.
18 is a flowchart of a process of learning using the improved gradient descent method.
19 shows an image processing result of a 3x3 kernel.
20 shows the effect of image processing using kernels of different sizes.
21 shows a result of processing an image using a one-dimensional kernel.
Fig. 22 shows the result of image test processing using the proposed joint processing method for 3x3 and 4x4 kernels.
Figure 23 shows the results of processing the image using three and four kernels.
24 shows image processing results using the proposed skip processing method.
25 shows an image processing result using the proposed complexity processing method.
26 shows a result processed using a polling layer with respect to a result processed using a convolutional layer.
27 shows a result of processing an image using a polling layer.
Figure 28 shows the results of the image processing test through the composite product / activation / polling layer method.
29 shows the results of image processing according to the quantity growth (3, 5, 8) of the composite product / activation using the proposed performance improvement process.
30 shows image processing results without using the proposed performance improvement process.
31 shows a learning process of a system (6 layers of deep learning) implemented using the proposed performance improvement task.
32 shows a learning process of a system that does not use the proposed performance improvement process.

Advantages and / or features of the present invention and methods for achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, only the present embodiments to make the disclosure of the present invention complete, and common knowledge in the art to which the present invention pertains. It is provided to fully inform the person having the scope of the invention, which is defined only by the scope of the claims. Like reference numerals refer to like elements throughout.

In addition, preferred embodiments of the present invention to be carried out below are provided in each system functional configuration to efficiently describe the technical components constituting the present invention, or system functions that are commonly provided in the technical field to which the present invention belongs. The configuration is omitted as much as possible, and will be described mainly on the functional configuration that should be additionally provided for the present invention. If those skilled in the art to which the present invention pertains, it will be easy to understand the functions of the components that are used in the prior art among the omitted functional configuration not shown below, and also the configuration omitted as described above The relationship between the elements and the components added for the present invention will also be clearly understood.

In addition, in the following description, the terms "transfer", "communication", "transmit", "receive", and the like in a signal or information mean that a signal or information is directly transmitted from one component to another component. As well as passing through other components. In particular, "transmitting" or "sending" a signal or information to a component indicates the final destination of the signal or information and does not mean a direct destination. The same is true for the "reception" of a signal or information.

Before describing embodiments of the present invention, a learning model structure based on CNN (Convolutional Neural Network) will be described.

The basic structure of the proposed learning model is shown in FIG. In this learning model, each group (210, 220, 230) is constructed as a 3 + 1 group of composite product and pooling layers, and each group (210, 220, 230) is performed three times in order, and then Affine layer 240 and Softmax layer 250 are performed. That is, the deep size of the basic structure of the proposed learning model has a (3 + 1) x 3 hidden layer. Here, the convolutional product hierarchy is a configuration corresponding to the convolution product hierarchy unit 110 of FIG. 1, and the pooling layer is a configuration corresponding to the pooling hierarchy unit 115 of FIG. 1.

The proposed learning model has the advantage of having scalability and descriptor simplicity with respect to the learning model in that groups are used instead of hidden layers. That is, when designing new models and functions based on the basic structure of the proposed learning model, new groups can be added or new function functions and structures can be added within the groups.

The reinforcement structure of the proposed learning model adds two convolutional layers 310 and 320 to each group based on the basic structure as shown in FIG. 3, and uses a small size kernel instead of a large size kernel for each group. do. The complex structure of the proposed learning model has a more complex structure and function than the simple group structure of the learning model described above, and designs a complex structure between groups.

The complex structure of the proposed learning model is shown in FIG. This composite structure has a group with an inception layer 410 and a skipped connection layer 420. The function of the inception layer 410 portion of the group processes the input data using various size kernels at the same time and outputs the results of the parallel connection for the various results. The function of the skipped connection layer 420 performs account change operations on the values of other groups and inceptions.

In particular, in the complex structure of the proposed learning model, it is possible to reduce the amount of computation and increase the computational speed according to the dip of the proposed learning model by using the 1x1, 3x3 and 3x1, 1x3 kernel formats. In addition, the skip connection layer 420 of the proposed learning model may include an inception layer 410, and the purpose of including a convolutional layer group is to add an extension capability of the proposed model. That is, the function of the skip connection layer 420 and the inception layer 420 may be modified when modifying the structure of the proposed learning model according to the purpose of learning. For reference, the inception layer 410 is a configuration corresponding to the inception layer unit 145 of FIG. 1, and the skip connection layer 420 is a configuration corresponding to the skip connection layer unit 150 of FIG. 1. .

Hereinafter, with reference to the accompanying drawings will be described embodiments of the present invention;

1 is a block diagram illustrating a composite product neural network based image processing system according to an exemplary embodiment of the present invention.

1, a convolutional neural network based image processing system 100 according to an exemplary embodiment may include a convolutional layer 110, an activation layer 115, a pooling layer 120, and a classification output layer. Unit 125, loss calculation layer 130, backward calculation unit 135, classification learning unit 140, inception layer unit 145, skip connection layer unit 150, parameter initial value design unit 155 ), A mini batch normalization design unit 160, a gradient descent optimization design unit 165, and a controller 170.

The composite product hierarchy 110 may extract a feature value of an object from input data using a plurality of kernel filters. Here, the plurality of kernel filters may be the same size or different sizes, some may be the same and some may be a different size, and the like. In addition, the plurality of kernel filters may be larger than a preset reference size or may be smaller than the reference size.

The activation layer unit 115 may perform a conversion operation on the feature value extracted by the composite product layer unit 110 using a nonlinear activation function. In this case, the activation layer 115 may perform a conversion operation on the feature value by using a rectified linear unit (RELU) activation function.

In the present embodiment, since the derivatives of the sigmoid function and the hyperbolic tangent function in the error backpropagation method have an inclination that the slope is close to 0 when the input value is increased, the deep learning does not proceed. It is preferable to use the reLU activation function, which has a high calculation speed and parallel processing speed, as the nonlinear activation function.

The pooling layer unit 120 may reduce the dimension, remove and suppress the noise by using a maximum polling operation on the output value of the activation layer unit 115. For reference, in the image recognition field, the maximum value may be extracted from the limited region of the image and the noise of the data may be mainly removed using the maximum value pooling operation.

As described above, the composite product neural network based image processing system 100 according to an exemplary embodiment removes noise of an image and extracts a feature through the composite product layer 110 and the pooling layer 120. Hereinafter, the composite product hierarchy 110 is calculated using a filter calculation method. 5 shows a 2x2 sized result 530 that is processed using a 3x3 sized filter 520 algorithm on an input 4x4 sized data 510. The filter operation process calculates the total after multiplying the windows corresponding to the input values at regular intervals.

Detailed calculation of the filter operation is performed at every place from A to B as shown in FIG. In addition, a padding operation is performed before and after performing a composite product operation on the image. The purpose of the padding operation is to set the size of the data to be processed and the size of the input data to be the same, and to set the size of the data to be fixed size (dimension) processed by a plurality of composite product operations.

There are two ways of calculating padding. The first fills a specific value around the input data. The second fills a certain value around the data being processed. As shown in the left figure of FIG. 7, one-width padding may be added to the size (3, 3) of the input data so that the data size (5, 5). It can be set to 1 in the actual situation and can be set to a desired integer according to the situation of the processing data. As shown in the right figure of FIG. 7, two-width padding may be added to the size (3, 3) of the input data so that the size (7, 7) of the data.

The purpose of the pooling layer 120 is to reduce the size of the dimension of the data being input and processed. In other words, the pooling operation reduces the size of the vertical and horizontal spaces in the data. There are various types of pooling operations (eg, average, median, maximum, minimum, etc.), and the pooling methods commonly used in embodiments of the present invention are a maximum value pooling operation and an average pooling operation. 8 illustrates a process of reducing the size of data using a 2 × 2 maximum value pooling operation. In particular, in the field of image recognition, the maximum value may be extracted from the limited region of the image and the noise of the data may be removed using a maximum polling operation.

Meanwhile, a composite product / pooling layer may be designed for multidimensional image processing. The composite product / pooling calculation can handle one-dimensional images and design a specific framework for efficiently processing multi-dimensional images. The framework performs a composite product / pooling operation on each dimension of the input multidimensional image in parallel and then outputs the multidimensional image.

9 illustrates a process of a multiplication / pooling framework processing a 3D image. This process is divided into three steps. In the first step, we extract each dimensional structure of a three-dimensional image. In the second step, we perform composite product / pooling on each dimensional structure of the image. In the third step, the state of the three-dimensional image is restored. In particular, the convolution product / pooling framework may perform parallel processing on multi-dimensional image data, and perform convolution product / pooling on appropriate multi-dimensional image data without burden of memory.

On the other hand, the activation layer 115 performs a conversion operation on the output value of the composite product layer 110. There are two types of activation functions, one for linear activation and one for nonlinear activation. In the embodiment of the present invention, a function mainly used is a nonlinear activation function, but a linear activation function and a nonlinear activation function may be used together according to the proposed model.

The calculation process of the nonlinear activation function uses a minimum value set for the operation result of the composite product hierarchy 110 to prevent a small value and to output a large value exceeding the minimum value. Non-linear activation functions mainly used in one embodiment of the present invention are sigmoid, hyperbolic tangent function, rectified linear unit (ReLU) function.

However, in the error backpropagation method, the derivatives of the sigmoid function and the hyperbolic tangent function do not proceed deep learning because the slope is close to 0 when the input value increases. For the above reason, it is preferable to use the ReLU activation function in one embodiment of the present invention. Especially in deep learning, ReLU function has faster learning speed, gradient calculation speed, and parallel processing speed than other functions.

Referring back to FIG. 1, the classification output layer unit 125 performs a classification prediction value on the input data (“Y1, Y2, FIG. 12” through an omnidirectional operation using the output value of the pooling layer unit 120). ..., Yn ").

The loss calculation layer unit 130 may calculate the loss value corresponding to the error value by comparing the classification prediction value with a predetermined target value (see "T1, T2, ..., Tn" in FIG. 12). . That is, the loss calculating layer unit 130 may calculate a difference between the classification prediction value and the target value as a loss value.

The reverse calculation unit 135 may obtain a correction value for the parameters of the hierarchical units 110, 115, 120, 125, and 130 by calculating a partial derivative value of the loss value through a reverse operation. .

Specifically, the backward calculation unit 135 calculates a partial differential value for the first parameter of the classification output hierarchy 125 from the loss calculation hierarchy 130 toward the classification output hierarchy 125. The partial output value for the second parameter toward the last hidden layer 110, 115, 120 may be calculated by the classification output layer unit 125. The backward calculation unit 135 calculates a partial differential value for the third parameter between the hidden layers 110, 115, 120, and the parameters of the hierarchical units 110, 115, 120, 125, and 130. It is possible to obtain a correction value for the variable (the first to third parameters).

12 schematically illustrates a calculation graph of the backward algorithm of the backward calculation unit 135 of FIG. 1. In FIG. 12, assuming a neural network performing a classification problem, an input value from the previous layer (composite product / pooling layer) is A (An is an input value of the nth layer, Wn is a parameter of the nth layer, and An + The calculation result of 1 represents WnAn schematically, and the output value of the classification output layer 1210 (softmax layer) is Y (Y1, Y2, ..., Yn: Yn is estimated for the nth target. Value). In particular, the result of Y is of type WnAn. In addition, the correct value (target value) for the input value is T (T1, T2, ..., Tn), and the loss calculation layer 1220 outputs the loss L using the output value. Loss L is calculated using Equation 1. The inverse calculation is partial differentiation for the loss L to obtain the parameter correction value of each layer of the learning model. For reference, the classification output layer 1210 is a configuration corresponding to the classification output layer unit 125 of FIG. 1, and the loss calculation layer 1220 is a configuration corresponding to the loss calculation layer unit 130 of FIG. 1. .

[Equation 1]

The reverse algorithm can be divided into three steps. In the first step, the partial differential values for the parameters of the classification output layer 1210 are calculated from the loss calculation layer 1220 to the classification output layer 1210. That is, the modified value of the parameter can be calculated using Equation 2.

[Equation 2]

In the second step, Equation 3 is used to calculate the partial differential value for the parameter from the classification output layer 1210 to the last hidden layer.

[Equation 3]

In the third step, Equation 4 is used to calculate partial differential values for the parameters between the hidden layers.

[Equation 4]

The detailed backward calculation process is shown in FIGS. 13 to 15. That is, FIG. 13 is a diagram illustrating in detail a partial differential calculation process for a first stage loss value, and FIG. 14 is a diagram illustrating in detail a partial differential calculation process for a parameter of a second stage hidden layer. The third step is a detailed illustration of the partial differential calculation process for the parameters of the hidden layers.

Referring back to FIG. 1, the classification learning unit 140 may perform an update on the parameter through a gradient descent method using the correction value and the learning rate derived through a certain amount of learning data. Can be.

The backward calculation can calculate the correction value for the parameters of the learning model by the partial differential method, but the meaning of this value gives only the direction of the correction for the parameter. So this value cannot tell how much to modify the parameter. In order to improve this problem, the classification learning unit 140 may calculate the update value of the parameter of the learning model using the gradient descent method.

The gradient descent process may calculate an update parameter based on a correction value and a learning rate calculated using predetermined amount of learning data. In particular, in the backward calculation of the process, the correction value is calculated using a batch method. A detailed process is as shown in FIG. 16, and the gradient descent method can update each parameter by using Equation 5.

[Equation 5]

In addition, there are two arrangement methods mainly used among the gradient descent methods, one method using one learning data, and another method using all learning data. The gradient descent can use both of these arrangements, but can randomly use a small amount of training data in consideration of insufficient memory problems (all training data placement states) and long learning times (Equation 6). Reference).

[Equation 6]

The quantity of learning data used in this embodiment is designed with a power of two. That is, the quantity of learning data for one time is 64 (26) to 512 (29), and higher quantity of learning data can be used according to the quantity of learning data.

Referring back to FIG. 1, the inception layer unit 145 performs a connection calculation in parallel using a nonlinear activation function on two or more feature values output by the convolutional layer layer 110. The parallel connection calculation result is performed to obtain a multi-dimensional result value, and the feature of the input data is combined with each other by performing a dimensional transform operation using a kernel filter and a linear activation function on the multi-dimensional result value. New feature values can be extracted.

The skip connection layer unit 150 generates multidimensional data through the inception layer unit 145 and the convolutional layer layer unit 110 with respect to the input data, and matches the input to match the number of dimensions of the multidimensional data. You can convert the number of dimensions of data. The skip connection layer unit 150 may suppress the feature loss problem of the input data by using a linear activation function after combining the multi-dimensional data and the input data whose dimension number is converted.

FIG. 10 illustrates a structure of the inception layer unit 145 of FIG. 1. The calculation process of the inception layer unit 145 includes three steps. In the first step 1010, the input data is processed in parallel using two or more different size filter convolutional layers and a nonlinear activation function. In the second step 1020, a matching operation (eg, a CONCAT function) is performed on each processing result of the first step 1010, and a multidimensional result is obtained. In a third step 1030, a dimensional transformation operation (dimension zooming of data) is performed on the multidimensional result of the second step 1020. In particular, the inception layer unit 145 uses various size filters and activation functions according to the calculation process.

In the first step 1010 of the inception layer unit 145, a size filter nonlinear activation function such as 1x1 and 3x3 is mainly used, and in the third step 1030, a 1x1 size filter and a linear activation function are mainly used. Use In addition, the inception layer unit 145 may extract various features of the input data in parallel with different size filters, and may be a new feature in which the features are combined with each other.

11 illustrates a structure of the skip connection layer unit 150 of FIG. 1. The calculation process of the skip connection layer unit 150 includes three steps. In a first step 1110, multidimensional data is generated using a composite product / pooling layer and an inception layer on input data. In the second step 1120, the number of dimensions of the input data is converted. The number of dimensions to be transformed then matches the number of dimensions of the multidimensional data of the first step 1110. In the third step 1130, after performing a matching operation (eg, an add operation) on the result of the first step 1110 and the result of the second step 1120, the multi-dimensional data is output using the linear activation function.

An advantage of the skip connection layer unit 150 is a skip operation. That is, the skip connection layer unit 150 may join the skip process and suppress the problem of feature loss of the training data according to the deepness of the proposed learning model. In addition, the skip connection layer unit 150 may include a structure of the inception layer unit 145 as well as the composite product layer unit 110. The advantage of the above-described structure is that it is possible to extract, combine and optimally learn various features of the data through three methods such as dip, width (features are related to each other), and skip over the data, rather than a simple dip model extraction method.

Referring back to FIG. 1, the parameter initial value design unit 155 may initialize the He based on the sample standard deviation calculated using the quantity of the parameters of the hierarchical units 110, 115, 120, 125, and 130. A value algorithm can be designed and initialization of the parameter can be performed based on the He initial value algorithm.

When the learning system is trained through the backward algorithm of the backward calculation unit 135 and the gradient descent method of the classification learning unit 140, an initial value setting method for a parameter of the learning system mainly uses a random initial value method. Can increase general-purpose performance, but cannot deal with parameter loss and expressiveness issues

In order to suppress the parameter loss and expressive power limitation problem, the learning system seeks a uniform distribution mapping rather than a parameter concentrated distribution mapping. Establishing Parameter Equal Distribution Mapping for Learning System The proposed learning model performs parameter initialization based on He initial value algorithm. He initial value method is designed based on Equation 7. In Equation 7, stddev represents a sample standard deviation and dpper_size represents a quantity of parameters of each layer of the learning model.

[Equation 7]

When the initial value of He is used, the parameter distribution remains uniform even when the layer is deepened. Thus, the parameter value is uniformly maintained in the learning model of nine or more layers, and the problem of parameter loss is suppressed.

Referring back to FIG. 1, the mini batch normalization design unit 160 may implement a mini batch for each of the input data and calculate an average value of the mini batch. The mini batch normalization design unit 160 calculates a variance value of the mini batch by using the average value of the mini batch, and uses the average value and the variance value of the mini batch for each input value of the mini batch. Each value of the normalized mini batch can be obtained.

Here, each value of the normalized mini-batch may be used as the input data in the composite product hierarchy 110.

We can suppress the parameter loss problem of the learning model by using the He initial value for problems such as parameter loss, but there is an overfit problem when learning according to the learning model dip. To compensate for this problem, the mini batch normalization design unit 160 uses a data mini batch normalization algorithm. The advantage of data mini-batch normalization is that it not only suppresses the overfitting problem but also speeds up learning.

Mini-batch normalization is the process of calculating the normalized value with the average of 0 and variance 1 of the input data. Specifically, the calculation process consists of three steps as shown in Equation (8). Referring to Equation 8, the first step is implemented in the form of mini-batches (x1, x2, ..., xn) for the n input data and calculates the average value (M) of the mini-batches. In the second step, the variance value (D) of the mini batch is calculated. The final step is to find each value (X1, X2, ..., Xn) of the normalized mini-batch using the mean value (M) and the variance value (D) for each input value in the mini-batch. In Equation 8, K is defined as a small value other than zero to prevent a situation of dividing by a small zero.

[Equation 8]

17 is a flowchart of a process of learning using mini-batch normalization.

Referring to FIG. 17, in operation 1710, an input value for input data is input, and in operation 1720, a composite product / pooling layer operation is performed to extract and output a feature value for the input value. Next, in step 1730, data mini-batch normalization is performed, in step 1740, an activation layer operation is performed, and in step 1760, the result of the activation layer operation is output as an output value. On the other hand, in step 1750, the gradient descent method is performed on the result of the activation layer operation to correct the parameter, and the correction value is transferred to the composite product / pooling layer of step 1720.

Referring back to FIG. 1, the gradient descent optimization design unit 165 calculates the momentum of the parameter, calculates a correction value of the learning rate using the derivative value of the parameter, and calculates the momentum and the learning rate. The correction value may be used to modify the parameter.

The calculation process of the gradient descent method is easy to implement and design, but this calculation process has an inefficient problem. In order to improve this inefficient problem, the gradient descent optimization design unit 165 designs an improved gradient descent method based on the Adam algorithm. The central idea of the Adam algorithm is to adjust for both methods based on the gradient descent method. The first way is to add momentum. In other words, the parameter value can be calculated by acceleration using random momentum each time. The second method is to adjust the learning rate. That is, the learning rate value is calculated by using the falling value of the parameter and the momentum each time.

The improved gradient descent method is obtained through a process of calculating an effective weight using Equation 9 on the basis of Adam, and may include three steps. In the first step, we calculate the momentum of the parameter (M variable in Equation 9). In the second step, the derivative value of the parameter is used to calculate the modified value of the learning rate (V variable in Equation 9). In the final step, the parameters of the learning model are modified using the modified values of momentum and learning rate. In Equation 9, a small value (K variable) is added to this value to prevent dividing the modified value of the learning rate (V variable) by zero.

[Equation 9]

18 is a flowchart of a process of learning using the improved gradient descent method.

Referring to FIG. 18, in operation 1810, an input value for input data is input, and in operation 1820, a composite product / pooling layer operation is performed to extract and output a feature value for the input value. Next, in step 1830, data mini-batch normalization is performed, in step 1840, an activation layer operation is performed, and in step 1860, the result of the activation layer operation is output as an output value. Meanwhile, in step 1850, the parameter is modified by performing the gradient descent method on the result of the activation layer operation. In this case, the parameter is modified by adjusting the momentum and the learning rate by improving the gradient descent method in step 1860. A (improved) gradient descent method is performed, and the correction value of the parameter by performing the improved gradient descent method is transferred to the composite product / pooling layer of step 1820. As described above, according to the exemplary embodiment of the present invention, acceleration learning may be performed using the improved gradient descent method, and an optimal parameter may be calculated relatively to the time limit.

Referring back to FIG. 1, the controller 170 is a composite product neural network based image processing system 100 according to an embodiment of the present invention, that is, the composite product layer unit 110, the activation layer unit 115, The pooling layer unit 120, the classification output layer unit 125, the loss calculation layer unit 130, the reverse calculation unit 135, the classification learning unit 140, the inception layer unit 145 The overall operations of the skip connection layer unit 150, the parameter initial value design unit 155, the mini batch normalization design unit 160, and the gradient descent optimization design unit 165 may be controlled.

System implementation

An image processing result of the 3x3 kernel according to an embodiment of the present invention is shown in FIG. 19. The right part of FIG. 19 shows the result of separating the background and extracting the features of the object.

20 shows the effect of image processing using kernels of different sizes. Smaller kernels can extract a lot of features but have a slow performance problem. In contrast, large kernels perform faster by extracting features. In order to comprehensively consider the problems such as the quantity and performance efficiency of feature extraction, the proposed system implements a composite product layer using 1x1, 3x3, 5x5, and 7x7 kernels.

One-dimensional processing is implemented based on the One-Dimensional Convolution method. In the proposed method, image processing is performed by using 1-dimensional kernel of 1xN and Nx1 (N = 1, 2, 3, 4 ...) format.

21 shows a result of processing an image using a one-dimensional kernel. The reason for using the 1-D kernel in the proposed system is that it is not burdened by the hardware and memory limitations and can accurately extract the features of the image, and it can be quickly calculated according to the hardware and memory limitations. Depending on the performance of the platform you compute in this way, you can consider the format of the kernel. The proposed system mainly uses 1x3, 3x1, 1x7 and 7x1 sized one-dimensional kernels.

The joining process is implemented in two-dimensional and one-dimensional kernel combining methods. In order to implement the proposed join processing method, it is divided into two steps. In the first step, various image processing results are output by using two-dimensional and one-dimensional kernels. In the second step, the results of the first step are concatenated into one result. The image test processing result using the combination processing method proposed in the 3x3 and 4x4 kernels is shown in FIG. 22. In addition, when implementing the proposed combining process, we use 3-4 different size kernels to extract the image features in multiple angles (kernel size, etc.). Figure 23 shows the results of processing the image using three and four kernels.

Skip processing is implemented in a partial recursion method using a two- and one-dimensional kernel. Three steps are included to implement the proposed skip process. In the first step, image processing is performed using two- and one-dimensional kernels. In the second step, the original image is adjusted to the same number of dimensions as the result of the first step by using a 1x1 kernel. In the final step, the results of the first and second steps are matched. 24 shows image processing results using the proposed skip processing method.

Complexity processing is implemented by combining combining and skip processing. This implementation process involves two steps. In the first step, the skip processing structure is implemented. In the second step, the join processing structure is implemented and included in the skip processing structure of the first step. In addition, in the proposed complexity process, dimension adjustment and padding processes are performed at each step to solve problems such as dimension mismatch and reduction. 25 shows an image processing result using the proposed complexity processing method.

The proposed polling layer performs an optimization process on the features extracted from the convolutional product layer and reduces the input feature values. Among these, two methods are used to implement the optimization process and the reduction work. The first method is implemented using the maximum value calculation method, and the second method is implemented using the average calculation method. It also adds a filler function to solve the problem of padding of images. The left part of FIG. 26 shows the processing result using the composite product layer. The result processed using the polling layer for the result shows a reduced image as shown in the right part of FIG. 26 and accurately represents the detailed characteristics of the object.

FIG. 27 is a result of processing an image using a polling layer. The left figure of FIG. 27 is a result processed using a maximum value calculated polling layer, and a right figure shows a result processed through an average calculated polling layer. When comparing the two results, the maximum computation polling layer may extract features such as an outline of an object rather than the average computation polling layer. The proposed system uses the maximum computation polling layer.

The proposed activation function hierarchy is implemented based on the activation function algorithm. In the image recognition field learning model, we implement it using ReLU-based algorithm.

Figure 28 shows the results of the image processing test through the composite product / activation / polling layer method. The test results confirm that the features of most of the objects have been extracted accurately. Also, the part that does not contain the feature of the object disappears like the background.

FIG. 29 shows image processing results according to the quantity growth (3, 5, 8) of the composite product / activation using the proposed performance improvement process, and FIG. 30 shows the image processing results without using the proposed performance improvement process. Shows. As shown in FIG. 29, in the system to which the proposed performance improvement process is applied, most features of the image can be extracted more accurately than the existing system according to the answer of the learning model. On the other hand, the situation that does not use the proposed performance improvement process as shown in Figure 30 shows the disadvantage that a number of features of the image is lost as the quantity growth of the composite product / activation.

FIG. 31 shows the learning process of the system (6 levels of deep learning) implemented using the proposed performance improvement task (accuracy: accuracy, epoch: learning count / test count, train: accuracy of learning process, test: testing process (Figure Accuracy) Figure 32 shows the learning process of a system that does not use the proposed performance improvement process, comparing the two learning processes, the accuracy, overfitting and rapid convergence of the system using the proposed performance improvement process, etc. It can be seen that good results can be obtained in various ways.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For the convenience of understanding, a processing device may be described as one being used, but a person skilled in the art will appreciate that the processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as parallel processors.

The software may include a computer program, code, instructions, or a combination of one or more of the above, and configure the processing device to operate as desired, or process independently or collectively. You can command the device. Software and / or data may be any type of machine, component, physical device, virtual equipment, computer storage medium or device in order to be interpreted by or to provide instructions or data to the processing device. Or may be permanently or temporarily embodied in a signal wave to be transmitted. The software may be distributed over networked computer systems so that they are stored or executed in a distributed manner. Software and data may be stored on one or more computer readable recording media.

Method according to the embodiment is implemented in the form of program instructions that can be executed by various computer means may be recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CDROMs, DVDs, and magnetic-optical such as floppy disks. Media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine code, such as produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described by the limited embodiments and the drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different manner than the described method, or other components. Or even if replaced or replaced by equivalents, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the following claims.

110: composite product hierarchy
115: Activation layer
120: pooling layer part
125: classification output layer
130: loss calculation hierarchy
135: reverse calculation unit
140: classification learning unit
145: inception hierarchy
150: skip connection layer portion
155: parameter initial value design unit
160: mini batch normalization design
165: gradient descent optimization design
170: control unit

Claims (10)

A compound product hierarchy unit for extracting feature values of an object from input data using a plurality of kernel filters;
An activation layer unit which performs a transform operation on a feature value extracted by the composite product layer unit using a nonlinear activation function;
A pooling layer unit which reduces a dimension, removes and suppresses noise by using a maximum polling operation on an output value of the activation layer unit;
A classification output layer unit for outputting a classification prediction value for the input data through an omnidirectional operation using the output value of the pooling layer unit;
A loss calculating layer unit comparing the classification prediction value with a predetermined target value and calculating a loss value corresponding to the error value;
A reverse calculation unit calculating a partial differential value of the loss value through a reverse operation to obtain a correction value for a parameter of each layer unit; And
A classification learning unit for updating the parameter through a gradient descent method using the correction value and the learning rate derived through a certain amount of learning data.
Synthetic product neural network based image processing system comprising a.
The method of claim 1,
The activation layer unit
and a transform product for the feature value using a reLU activation function.
The method of claim 1,
The reverse calculation unit
Calculate a partial differential value for the first parameter of the classification output hierarchy in the loss calculation hierarchy towards the classification output hierarchy and for a second parameter in the direction of the last hidden hierarchy in the classification output hierarchy. A partial differential value is calculated, and a partial differential value for a third parameter between hidden layers is calculated to obtain a correction value for a parameter (the first to third parameters) of each layer part. Synthetic product neural network based image processing system.
The method of claim 1,
Perform a linkage calculation in parallel using a nonlinear activation function on two or more feature values output by the convolutional product layer, and perform a matching operation on the parallel linkage calculation results to obtain multidimensional result values. An inception layer unit for extracting a new feature value combining features of the input data by performing a dimensional transformation on the multi-dimensional result using a kernel filter and a linear activation function
Synthetic product neural network based image processing system further comprising.
The method of claim 4, wherein
Generate multidimensional data for the input data through the inception layer unit and the composite product layer unit, convert the number of dimensions of the input data to match the number of dimensions of the multidimensional data, and convert the multidimensional data and the number of dimensions A skipped connection layer unit for suppressing a feature loss problem of the input data using a linear activation function after matching the input data
Synthetic nerve-only image processing system further comprises a.
The method of claim 1,
A parameter initial value design unit for designing a He initial value algorithm based on a sample standard deviation calculated using the quantity of parameters of each layer unit, and performing initialization of the parameter based on the He initial value algorithm.
Synthetic nerve-only image processing system further comprises a.
The method of claim 1,
Each input data is implemented in the form of a mini batch, calculates an average value of the mini batch, calculates a variance value of the mini batch using the average value of the mini batch, and inputs each input value of the mini batch. Mini batch normalization design unit for calculating each value of the normalized mini batch using the mean value and the variance of the mini batch
More,
The composite product layer part
And using each value of the normalized mini-batch as the input data.
The method of claim 1,
Gradient descent method for calculating the momentum of the parameter, calculating the correction value of the learning rate using the derivative value of the parameter, and performing the correction operation of the parameter using the momentum and the correction value of the learning rate Optimization design department
Synthetic nerve-only image processing system further comprises a.
In the image processing method using a composite product neural network based image processing system,
Extracting a feature value of an object from input data by using a plurality of kernel filters of the image processing system by using a plurality of kernel filters;
Performing, by the activation layer unit of the image processing system, a transform operation on the feature value extracted by the composite product layer unit using a nonlinear activation function;
A pooling layer unit of the image processing system reduces the dimension, removes and suppresses the noise by using a maximum polling operation on an output value of the activation layer unit;
Outputting, by the classification output layer unit of the image processing system, a classification prediction value for the input data through an omnidirectional operation using the output value of the pooling layer unit;
A loss calculating layer unit of the image processing system comparing the classification prediction value with a predetermined target value and calculating a loss value corresponding to the error value;
A backward calculation unit of the image processing system calculates a partial differential value of the loss value through a backward operation to obtain a correction value for a parameter of each layer unit; And
The classifying learning unit of the image processing system performs an update on the parameter through a gradient descent method using the correction value and the learning rate derived through a certain amount of learning data.
Composite product neural network based image processing method comprising a.
The method of claim 9,
Calculating, by the gradient descent optimization design unit of the image processing system, momentum of the parameter;
Calculating, by the gradient descent optimization design unit, a correction value of the learning rate using the derivative value of the parameter; And
Performing a correction operation of the parameter by using the gradient descent optimization design unit by using the correction value of the momentum and the learning rate
Synthetic nerve-only image processing method further comprising a.

Images