KR102042446B1 - Improved binarization apparatus and method of first layer of convolution neural network - Google Patents

Abstract

An improved binarization apparatus and method for a first layer of a convolutional neural network. The improved binarization apparatus for a first layer of a convolutional neural network according to an embodiment of the present disclosure generates binarization input data by binarizing initial input data for the layer. A binarization parameter generator for generating binarization parameters by binarizing the parameters of the first layer, performing a multiplexing on the binarization parameters to generate a multiplexing parameter, and performing a convolution operation between the binarization input data and the multiplexing parameter A convolution operation unit that generates a first operation result and a short operation unit that calculates a second operation result by adding a shortcut to the first operation result, wherein the shortcut may correspond to the initial input data before binarization. .

Description

IMPROVED BINARIZATION APPARATUS AND METHOD OF FIRST LAYER OF CONVOLUTION NEURAL NETWORK

The present application is directed to an improved binarization apparatus and method of the first layer of a convolutional neural network.

Deep Neural Networks (DNNs) have shown outstanding performance in many fields, including computer vision and speech recognition. Especially, in the field of computer vision, the Convolutional Neural Network (CNN) has high image analysis performance. Showed. To improve the CNN's analytical performance, a large number of artificial nerves must be included in the model, which requires very high computational complexity. In addition, a very large memory capacity is required to implement a connection relationship between numerous artificial nerves.

However, due to CNN's high memory demands and computations, it works well in environments with expensive, high-performance, high-power graphics processing units (GPUs), but it is difficult to operate quickly on embedded platforms such as smartphones and smart wearable devices. . As a result, there is a demand for a method for reducing the amount of computation and memory requirements of the CNN.

Background art of the present application is disclosed in Korea Patent Publication No. 2017-0023708.

The present invention is to solve the above-mentioned problems of the prior art, an object of the present invention to provide an improved binarization apparatus and method of the first layer of the convolutional neural network that can minimize the loss of input data while binarizing both the convolution input data and parameters. do.

The present invention is to solve the above-mentioned problems of the prior art, an improved binarization device of the first layer of the convolutional neural network that can reduce the amount of floating point multiplication and reduce the degradation of analysis performance compared to the conventional convolutional neural network operation And to provide a method.

However, the technical problem to be achieved by the embodiments of the present application is not limited to the technical problems as described above, and other technical problems may exist.

As a technical means for achieving the above technical problem, the improved binarization method of the first layer of the convolutional neural network according to an embodiment of the present application, (a) binarizes the initial input data for the first layer to generate the binarized input data Generating a binarization parameter by binarizing the parameter of the first layer, (b) performing a binarization on the binarization parameter to generate a binarization parameter, and performing a convolution operation between the binarization input data and the binarization parameter. Performing multiplication multiplication with the multiplication factor for the convolution operation to generate a first operation result, and (c) calculating a second operation result by adding a short cut to the first operation result. Wherein the shortcut corresponds to the initial input data before binarization Can.

According to one embodiment of the present application, by the addition operation of the shortcut in the step (c), the initial input data before the binarization in the step (a) can be considered.

According to an embodiment of the present disclosure, in the step (b), the first operation result is batch normalized, and in the step (c), the shortcut may be batch normalized.

According to one embodiment of the present application, the quadratic operation result may be calculated by Equation 1.

According to an embodiment of the present disclosure, the improved binarization method of the first layer of the convolutional neural network may further include (d) performing pooling and batch normalization on the quadratic result.

The improved binarization apparatus of the first layer of the convolutional neural network according to an embodiment of the present invention binarizes initial input data for the layer to generate binarization input data, and binarizes the parameter of the first layer to generate binarization parameters. A generation unit generates a doubling parameter by performing a doubling on the binarization parameter, performs a convolution operation between the binarization input data and the doubling parameter, and then multiplies a multiplication factor by the doubling parameter with respect to the convolution operation. A convolution operation unit configured to generate a first operation result and a short cut operation unit configured to calculate a second operation result by adding a shortcut to the first operation result, wherein the short cut may correspond to the initial input data before binarization. have.

According to one embodiment of the present application, by the addition operation of the shortcut, the initial input data before binarization may be considered.

According to the exemplary embodiment of the present application, the convolution operation unit may perform batch normalization of the first operation result, and the shortcut operation unit may perform batch normalization of the shortcut.

According to the exemplary embodiment of the present application, the short cut operation unit may calculate the secondary calculation result through Equation 2.

According to an embodiment of the present disclosure, the improved binarization apparatus of the first layer of the convolutional neural network may further include a normalization operation unit that performs pooling and batch normalization on the second operation result.

The above-mentioned means for solving the problems are merely exemplary and should not be construed as limiting the present application. In addition to the above-described exemplary embodiments, additional embodiments may exist in the drawings and detailed description of the invention.

According to the above-described problem solving means of the present invention, an improved binarization apparatus of the first layer of the convolutional neural network that can minimize the loss of input data by adding a shortcut operation that bypasses initial input data while binarizing both convolution input data and parameters. And methods.

The present invention is to solve the above-described problems of the prior art, by binarizing the initial input data and the parameters of the first layer to reduce the amount of floating point multiplication compared to the operation of the conventional convolutional neural network, analysis performance through the short cut operation It is possible to provide an improved binarization apparatus and method of the first layer of the convolutional neural network, which can reduce the degradation of.

1 is a diagram illustrating a process of a conventional general convolutional neural network.
2 is a diagram illustrating the configuration of an improved binarization apparatus of a first layer of a convolutional neural network according to an embodiment of the present disclosure.
3 is a diagram illustrating an improved binarization flow of a first layer of a convolutional neural network according to an embodiment of the present disclosure.
4 is a diagram illustrating an analysis performance reduction by an improved binarization apparatus of a first layer of a convolutional neural network according to an embodiment of the present disclosure.
5 is a diagram comparing the amount of computation of the improved binarization apparatus of the first layer of the convolutional neural network and the conventional convolutional neural network according to an embodiment of the present application.
6 is a diagram comparing the analysis performance of the conventional CNN and the improved binarization device of the first layer of the convolutional neural network according to an embodiment of the present application.
FIG. 7 illustrates a flow of an improved binarization method of a first layer of a convolutional neural network according to an embodiment of the present disclosure.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present disclosure. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In the drawings, parts irrelevant to the description are omitted for simplicity of explanation, and like reference numerals designate like parts throughout the specification.

Throughout this specification, when a portion is "connected" to another portion, this includes not only "directly connected" but also "electrically connected" with another element in between. do.

Throughout this specification, when a member is said to be located on another member "on", "upper", "top", "bottom", "bottom", "bottom", this means that any member This includes not only the contact but also the presence of another member between the two members.

Throughout this specification, when a part is said to "include" a certain component, it means that it can further include other components, without excluding the other components unless specifically stated otherwise.

1 is a diagram illustrating a process of a conventional general convolutional neural network.

A conventional general convolutional neural network (hereinafter, referred to as a conventional CNN) is a plurality of convolutional layers (hereinafter, referred to as "features of an image") that largely extract input local characteristics. CL) and a plurality of fully-connected layers (FCLs) for analyzing the extracted characteristics. Information input to the conventional CNN may be classified through a plurality of CL and FCL. For example, if the inputted information is an image, it may be classified into which category the image belongs to according to the feature point of the image. Precise 32-bit floating-point values of CNN's parameter and convolution inputs are known to be insignificant for high image analysis performance, and research has been conducted to represent parameter and convolution inputs with lower bit counts. Referring to FIG. 1A, a conventional CNN includes a process of convolution (S11), pooling (S12), batch normalization (S13), and activation (S40). The convolution S11 is an operation between the input data and the plurality of filters, and the parameters of each filter can be obtained by learning W x h x d of input A and (2n-1) x (2n). The result O of the convolution between M filters W having a size of −1) × d may be expressed by Equation 1 below.

[Equation 1]

는m 번째 필터

의 (x, y, z) 위치에서의 파라미터 값이고 A _x,y,z 는 A의 (x, y, z) 위치에서의 성분 값이다 상기 수학식1에서와 같이 종래의 CNN에서 O의 성분 값을 계산하기 위해서는 (2n-1)·(2n-1)·d번의 곱셈-누산이 필요하며, O의 크기는 (w-2n+2) Х(h-2n+2)ХM이 된다. 이에 따라, 하나의 CL을 처리하기위해 총 (2n-1)·(2n-1)·d·(w-2n+2)·(h-2n+2)·M번의 연산이 요구되며, (2n-1)·(2n-1)·d·M개의 파라미터를 저장할 메모리가 필요하다. 또한, 컨벌루션 연산 후에는 채널별로 풀링을 수행하고(S20), O를 배치 정규화한다(S30). 풀링은 국부 영역 내의 값들에 대한 평균 또는 최대값을 선택하여 입력데이터의 크기를 줄이는 것을 의미한다. 이후 비선형 활성화 연산을 통과한 활성화 출력은 다음 레이어의 입력으로 사용된다. 여기서, 비선형 활성화 연산은 비선형 함수를 이용한 연산을 의미하며, 비선형 함수는, 예를 들어, 시그모이드(sigmoid) 함수나 정류 선형 함수(ReLU : Rectified Linear Unit)를 포함할 수 있다. 또한, 활성화 출력은 상기 비선형 함수를 이용한 연산의 결과를 의미한다. The resulting value O represents the component value at position (i, j, m). Also,

Mth filter

Is the parameter value at position (x, y, z) of A and _{x, y, z} is the component value at position (x, y, z) of A. To calculate the value, (2n-1) · (2n-1) · d times of multiplication-accumulation is required, and the magnitude of O is (w-2n + 2) Х (h-2n + 2) ХM . Accordingly, a total of (2n-1), (2n-1), d, (w-2n + 2), (h-2n + 2), and M operations are required to process one CL, and (2n -1) A memory for storing (2n-1) d d M parameters is required. In addition, after the convolution operation, pooling is performed for each channel (S20), and O is batch normalized (S30). Pulling means reducing the size of the input data by selecting an average or maximum value for the values in the local area. The activation output, which then passes the nonlinear activation operation, is used as the input for the next layer. Here, the nonlinear activation operation may mean an operation using a nonlinear function, and the nonlinear function may include, for example, a sigmoid function or a rectified linear unit (ReLU). In addition, the activation output means the result of the operation using the nonlinear function.

The FCL refers to a process involving fully connected convolution operations, group normalization and activation. The fully connected convolution performs the same operation as the convolution operation of the CL, with M filter sizes equal to the input size, and the output size is 1 × 1 μM . Accordingly, in order to process one FCL, a multiplication-accumulation operation of w · h · d · M times is required, and a memory for storing w · h · d · M parameters is required. As described above, in order to implement a conventional CNN, a large amount of computation and a large memory capacity are required. Due to these problems, it is difficult to implement a conventional CNN on a platform having limited associative performance and memory capacity, such as an embedded system.

In addition, conventionally, a CNN has been proposed a method of expressing both a parameter and a convolution input in 8-bit floating point, and a method of expressing a parameter in 3 values and a convolution input in 3 bits has also been proposed. In addition, a method of dramatically reducing the amount of computation and memory required by the CNN through a binarization process represented by an extreme single bit (BCNN) has been proposed as shown in FIG. Referring to (b) of FIG. 1, the CNN for a conventional binarized input binarizes the input of a layer to perform convolution on the binarized input (S21), multiplexing (S22), pooling (S23), and arranging. It may include a process of normalization (S24) and binarization (S25). The binarized convolution S21 is an operation between the input and the plurality of filters, and the convolution input may be represented as a single bit as Equation 2 as an activation result A ^b of the convolution layer before binarization.

[Equation 2]

Where l is the layer number, clip () represents the max (-1, min (·, 1)) operation, and sign (·) represents the operation to extract (return) the sign. (A _l ) ^b _{x, y, z} represents the input of binarized convolution, and (A _l ) _{x , y , z} represents the input of convolution before binarization. In addition, unlike the general CNN, the parameters of each filter may be represented in a binary form W ^b rather than a floating point type W. The operation of binarizing the parameters of each filter may be represented as in Equation 3.

[Equation 3]

Here, (W ₁ ) ^b _{m, x, y, z} represents a binarized parameter, and (W ₁ ) _{m, x, y, z} represents a parameter before binarization. The convolution operation between the input and the parameter of the binary convolution having only a value of -1 or 1 may be replaced by Bitwise XNOR and accumulation, which will be described later. In addition, the result of the binarized convolution between the input A ^b of the size w ⅹ h ⅹ ^d and the M filters W ^b of the size (2n-1) ⅹ (2n-1) ⅹ ^d can be expressed as Equation 4.

[Equation 4]

Where l is the layer number, and W _{m, x, y, z} , W ^b _{m, x, y, z} are the floating-point parameter values and binarized at the (x, y, z) position of the m th filter, W _m , respectively. It means the parameter value. In addition, the multiplication accumulation between W ^b and A ^b may be replaced by a bitwise XNOR and accumulation between a parameter mapped to a single bit and a convolution input, as shown in Equation (5).

[Equation 5]

Here, XNOR (·, ·) is an XNOR operation between two inputs of a single bit, and has an output of -1 if the values of the two inputs are different, and 1 if they are the same. As in Equation 5, the size of O becomes wⅹhⅹM by changing only the number of channels in the case of zero-padding. Accordingly, a total of (2n-1)-(2n-1) -d-w-h-M bit operations are required to process the binary convolution of CL.

After the binarized convolution (S21), a multiplication may be performed by multiplying the parameter doubling coefficient e (S22). Drainage can be performed to reduce the reduction in analysis performance due to binarization of parameters. Thus, the multiplexing coefficient e can be introduced to approximate the binarized parameter to the parameter before binarization. The multiplexing coefficient may be defined as an average of absolute values of the corresponding channel parameter for each channel as shown in Equation 6. The doubling coefficient may be generated by the number of channels of the corresponding layer. In addition, as the multiplication is performed as shown in Equation 7, the binarized parameter value may be approximated to the parameter value before binarization.

[Equation 6]

Here, e _{l, m} represents a doubling coefficient.

[Equation 7]

과 분산

을 기록하고, 스케일 (Scale) 파라미터 γ_m와 바이어스 파라미터 β_m를 학습하여 입력을 정규화함으로써 공변량 이동 현상을 억제시킬 수 있다.That is, the doubling (S22) is an operation of multiplying each component of the binarized convolution result O by e, and requires a floating point multiplication operation of w · h · M times, followed by pooling (S23) and batch normalization (S24). , Activation consumes the same number of floating-point operations. Pooling (S23) is a process of reducing the size of a feature map through maximum value pooling after binarization convolution and multiplexing (S22). Equation 8 represents a batch normalization operation, and the input may be normalized in units of channels during the learning process. That is, the average in the channel unit of the input

And dispersion

The covariate shift phenomenon can be suppressed by normalizing the input by learning the scale parameter γ _m and the bias parameter β _m .

[Equation 8]

은 각각 m 번째 필터의 평균, 분산, 스케일링 파라미터, 바이어스 파라미터를 나타내며, 각각은 필터 수, 즉 입력의 채널 수 만큼의 개수를 가진다. 보편적으로 2ⅹ2, 혹은 3ⅹ3의 국부 영역 내의 값들의 평균값 또는 최댓값을 선택하여 입력의 크기를 (w/2)ⅹ(h/2)ⅹM 으로 줄인다. 이 후 전술한 수학식 2를 통한 이진화가 이루어지고(S25), 출력된 레이어는 다음 레이어의 입력으로 사용될 수 있다.Where X _{m, x, y, z} represents the input to the batch normalization process _,

Represents the average, variance, scaling parameter, and bias parameter of the m th filter, and each has the number of filters, that is, the number of channels of the input. Typically, the average or maximum value of the values in the local area of 2ⅹ2 or 3ⅹ3 is chosen to reduce the size of the input to (w / 2) ⅹ (h / 2) ⅹM. Thereafter, binarization is performed through Equation 2 (S25), and the output layer may be used as an input of a next layer.

If the image data passes through the CL several times and is greatly reduced in size due to a number of pooling operations, the final image analysis is performed through the FCL. The operation structure of FCL is the same as CL, but the operation corresponding to the convolution is replaced by the first-combination operation between the input and the parameter. To this end, the CL output of the three-dimensional form is converted into the one-dimensional form and convolved with a plurality of one-dimensional filters. Equation (9) represents the value of the mth component of the output O generated as a result of the operation between k sized binarized inputs A ^b and M equalized M sized filters W ^b .

[Equation 9]

Where W _{m, x} represents the m-th filter and the input, and A _x represents the x-th component value of the input. Equation 9 consumes a total of k · M bit operations.

Conventional BCNN has not binarized the input layer of the neural network that receives the image data as input due to the performance reduction due to data loss during the binarization. However, non-binarized CL's convolution operation requires a huge amount of floating point operations, which exceeds the sum of all the floating point operations required for batch normalization, pooling, and activation of each layer. The plan needs to be presented.

2 is a view showing the configuration of the improved binarization apparatus of the first layer of the convolutional neural network according to an embodiment of the present application, Figure 3 is a flow of the improved binarization of the first layer of the convolutional neural network according to an embodiment of the present application Figure is shown.

Referring to FIG. 2, the improved binarization apparatus 100 of the first layer of the convolutional neural network may include a binarization parameter generator 110, a convolution operator 120, a shortcut operator 130, and a normalization operator 140. have. Referring to FIG. 5, the parameter generator 110 may generate binarized input data by binarizing initial input data of a first layer. Since there is no previous layer in the first layer, the convolution input does not have a value of -1 or 1. Accordingly, the binarization parameter generator 110 may binarize the initial input data, that is, the convolution input, through Equation (10).

[Equation 10]

Where (A _l ) ^b _{x, y, z} is Binary convolution input of the first layer, and ( _Al ) _{x, y, z} represents the convolution input before binarization. When binarizing the first layer, the analysis performance due to loss of input image data is degraded, but convolution can be replaced by binary convolution to reduce memory requirements and computational complexity for storing parameters.

In addition, the binarization parameter generator 110 may generate a binarization parameter by binarizing the parameter of the first layer (S51). For example, the binarization parameter generator 110 may generate the binarization parameter by binarizing the parameter of the first layer through Equation (11).

[Equation 11]

Here, (W ₁ ) ^b _{m, x, y, z} represents the binarized parameter of the first layer, and (W ₁ ) _{m, x, y, z} represents the parameter before the binarization of the first layer.

The convolution calculator 120 may generate a multiplexing parameter by performing multiplexing on the binarization parameter (S52). For example, the convolution operator 120 may calculate a doubling factor through Equation 12, and generate a doubling parameter by performing multiplexing on the binarization parameter through Equation 13.

[Equation 12]

Here, e ₁ represents a doubling coefficient.

[Equation 13]

Here, ( W ₁ ) ^b _{m, x, y, z} is a binarized parameter, and ( W ₁ ) _{m, x, y, z} is a value obtained by approximating the binarized parameter to a parameter before binarization through a fold factor. Indicates.

The convolution operator 120 may generate a first operation result of performing a convolution operation between the binarization input data and the multiplexing parameter. The convolution operator 120 may generate the first operation result through Equation 14.

[Equation 14]

Here, ( O ₁ ) _{I, j, m} is the result of the first order operation, and ( A ₀ ) ^b means the binarized initial input data.

In addition, the convolution operation unit 120 may batch normalize the results of the first operation (S53). For example, the convolution operator 120 may perform batch normalization using Equation 8.

[Equation 8]

은 각각 m 번째 필터의 평균, 분산, 스케일링 파라미터, 바이어스 파라미터를 나타내며, 각각은 필터 수, 즉 입력의 채널 수 만큼의 개수를 가진다In the batch normalization process by the convolution operator 120, X _{m, x, y, z} may be an input of a batch normalization process, that is, a first order result (( O ₁ ) _{I, j, m} ). Also,

Represents the average, variance, scaling parameter, and bias parameter of the m-th filter, each having the number of filters, that is, the number of channels of the input.

When batch normalization is applied to the first operation result, it may be expressed as in Equation 15.

[Equation 15]

The short cut operation unit 130 may calculate a second arithmetic result by adding a short cut to the first arithmetic result. As described above, there is an advantage that the amount of computation can be reduced by binarizing the input data of the layer, but there is a problem that data is lost due to the binarization of the input data. Accordingly, a short cut operation that bypasses the initial input data before binarization and adds to the first result of the convolution operation to obtain a result close to the convolution without binarization of the input. In other words, the shortcut may correspond to initial input data before binarization. In addition, the initial input data before binarization may be considered by an addition operation of a shortcut.

The shortcut operator 130 may calculate a quadratic operation result through Equation 16.

[Equation 16]

Here, e _{1, m} is the coefficient of the m-th filter for the first layer, A ₀ is the initial input data, I, j, m represents the position component value of A ₀ .

Equation 16 represents an addition operation of the first input result (see Equation 14) and initial input data without batch normalization, which is not considered to suppress the covariate shift phenomenon. Accordingly, the shortcut operator 130 may perform batch normalization of the first order calculation result and normalize initial input data as shown in Equation 15 (S54), and perform the batch normalization first order result and the batch normalization shortcut. You can add operations.

The shortcut operator 130 may calculate a quadratic operation result through Equation 17. Equation 17 shows the result of the batch normalized first order operation and the addition operation of the batch normalized shortcut.

[Equation 17]

According to Equation 17, since the batch normalized first order result and the batch normalized shortcut are added, it is necessary to perform batch normalization again. The normalization operator 140 may perform pooling and batch normalization on the result of the quadratic operation. The normalization operator 140 may perform batch normalization on the result of the quadratic operation using Equation 8 (S55). In addition, the normalization operation unit 140 may activate the quadratic operation result (S56) and pool (S57). Activation, for example, means an operation using a nonlinear function, and the nonlinear function may include, for example, a sigmoid function or a rectified linear unit (ReLU). In addition, pooling means reducing the size of the input data by selecting an average or maximum value for the values in the local area. Since activation and pooling are well-known concepts, detailed descriptions thereof will be omitted.

Like conventional neural networks that perform learning with various ImageNet data sets, ResNet-18 reduces the image data to a size of 224 × 224 × 3 and then normalizes them by preprocessing. The first layer is a 7 × 7 convolution with a stride value of 2, with 64 7 × 7 × 3 parametric filters, with an output size of 112 × 112 × 64. Thus, the existing BCNN consumes 7x7x3x112x112x64 floating point MAC operations in the input layer convolution, and 112x112x64 floating point multiplication operations in the multiplexing, batch normalization, pooling, and activation operations.

On the other hand, the improved binarization apparatus 100 of the first layer of the convolutional neural network binarizes normalized image data to 1, -1 through sign operation, and then performs binarized convolution to calculate a first operation result and performs a sign operation. The second operation result may be calculated by adding the image data that is not performed, that is, the shortcut to the first operation result. For example, in order to match the size of the image data of the shortcut (224x 224x3) with the size of the binarized convolution output 112x112x64, subsamples were created at 2x2 size, and the RGB 3 channels were repeatedly connected to increase the number of 64 channels. , 7x7x3x112x112x64 floating point MAC operations are converted into XNOR bit operations and accumulation operations, thereby reducing the amount of computation compared to the conventional neural network.

Hereinafter, the effectiveness of the conventional binarized convolution and the convolutional neural network by the improved binarization apparatus 100 by the improved binarization apparatus 100 will be verified through experiments comparing the analysis performance, the floating-point operation amount, and the bit operation amount. To measure the analysis performance, we use the ImageNet dataset ILSVRC2012, a large color image dataset with an average size of 469 × 387. In addition, the training data is maintained to maintain the ratio of the image so that the smaller one of the width and height of the input image has a 256 size, and after the reduction, a simple color change is made and cut out to a size of 224 × 224 at an arbitrary position. For the test data, 224 × 224 size is cut out from the center of the image without any color change.

4 is a diagram illustrating an analysis performance reduction by an improved binarization apparatus of a first layer of a convolutional neural network according to an embodiment of the present disclosure.

Referring to FIG. 4, the BCNN binarizing the first layer has a 1-1% reduction in Top-1 analysis performance compared to the conventional CNN, but adds a shortcut by the improved binarization apparatus 100 of the first layer of the convolutional neural network. In the case of one convolution operation, the performance of Top-1 analysis is reduced by 3.8%, which is suppressed by 13.7% with the equivalent amount of computation.

5 is a diagram comparing the amount of computation of the improved binarization apparatus of the first layer of the convolutional neural network and the conventional convolutional neural network according to an embodiment of the present application.

FIG. 5 shows the sum of the floating point multiplication and bit operations of convolution, batch normalization, activation, and pooling per layer. Referring to FIG. 5, according to the improved binarization apparatus 100 of the first layer of the convolutional neural network, it can be seen that 94.53% of the floating point multiplication operations of the conventional BCNN are converted to bit operations.

In addition, in another experiment to measure the analytical performance of the improved binarization apparatus 100 of the first layer of the convolutional neural network, we use CIFAR10 as the data set, and the CNN model consists of six convolutional layers and three pulley connected layers. I used ConvNet. The CIFAR10 is a small 32x32 color image data set, which is composed of 50,000 training sets for 10 object classifications and 10,000 test sets.

6 is a diagram comparing the analysis performance of the conventional CNN and the improved binarization device of the first layer of the convolutional neural network according to an embodiment of the present application.

Referring to FIG. 6, it can be seen that the degradation of the analysis performance of the convolution by the improved binarization apparatus 100 of the first layer of the convolutional neural network is 10% suppressed compared to BCNN which binarized the first layer.

FIG. 7 illustrates a flow of an improved binarization method of a first layer of a convolutional neural network according to an embodiment of the present disclosure.

The improved binarization method of the first layer of the convolutional neural network according to the embodiment of the present application shown in FIG. 7 is the improved binarization apparatus of the first layer of the convolutional neural network according to the embodiment of the present application described with reference to FIGS. It can be performed by. Therefore, even if omitted below, the descriptions of the improved binarization apparatus of the first layer of the convolutional neural network according to an embodiment of the present disclosure through FIGS. 2 to 6 may be equally applicable to FIG. 7.

Referring to FIG. 7, in step S1010, the binarization parameter generator 110 generates binarization input data by binarizing initial input data of a first layer of a convolutional neural network, and generates binarization parameters by binarizing the parameters of the first layer. Can be. For example, the binarization parameter generator 110 may binarize initial input data, that is, a convolution input, through Equation 10 above.

[Equation 10]

Where (A _l ) ^b _{x, y, z} is Binary convolution input of the first layer, and ( _Al ) _{x, y, z} represents the convolution input before binarization.

In addition, the binarization parameter generator 110 may generate the binarization parameter by binarizing the parameter of the first layer through Equation (11).

[Equation 11]

Here, (W ₁ ) ^b _{m, x, y, z} represents the binarized parameter of the first layer, and (W ₁ ) _{m, x, y, z} represents the parameter before the binarization of the first layer.

In operation S1020, the convolution calculator 120 may generate a multiplexing parameter by performing a multiplexing on the binarization parameter, and generate a first operation result of performing a convolution operation between the binarization input data and the multiplexing parameter. . For example, the convolution operator 120 may calculate a doubling factor through Equation 12, and generate a doubling parameter by performing multiplexing on the binarization parameter through Equation 13.

[Equation 12]

Here, e _{1, m} represents a doubling coefficient.

[Equation 13]

Where ( W ₁ ) ^b _{m, x, y, z} is a binarized parameter, and e _{1, m} ( W ₁ ) ^b _{m, x, y, z} is a binarization factor before binarization. The parameter ( W ₁ ) represents a value approximated by _{m, x, y, z} .

The convolution operator 120 may generate the first operation result through Equation 14.

[Equation 14]

Here, ( O ₁ ) _{I, j, m} is the result of the first order operation, and ( A ₀ ) ^b means the binarized initial input data.

In addition, the convolution operator 120 may batch normalize the results of the first operation. For example, the convolution operator 120 may perform batch normalization using Equation 8.

[Equation 8]

In operation S1030, the shortcut operator 130 may calculate the secondary operation result by adding the shortcut to the primary operation result. In exemplary embodiments, the shortcut may correspond to initial input data before binarization. In addition, the initial input data before binarization may be considered by an addition operation of a shortcut.

The shortcut operator 130 may calculate a quadratic operation result through Equation 16.

[Equation 16]

Here, e _{1, m} is the coefficient of the m-th filter for the first layer, A ₀ is the initial input data, I, j, m represents the position component value of A ₀ .

Equation 16 represents an addition operation of the first input result (see Equation 14) and initial input data without batch normalization, which is not considered to suppress the covariate shift phenomenon. Accordingly, the shortcut operator 130 may perform batch normalization on the first order result and normalize initial input data as shown in Equation 15, and perform addition calculation on the batch normalized first order result and batch normalized shortcut. Can be.

The shortcut operator 130 may calculate a quadratic operation result through Equation 17. Equation 17 shows the result of the batch normalized first order operation and the addition operation of the batch normalized shortcut.

[Equation 17]

According to Equation 17, since the batch normalized first order result and the batch normalized shortcut are added, it is necessary to perform batch normalization again. The normalization operator 140 may perform pooling and batch normalization on the result of the quadratic operation. The normalization operator 140 may perform batch normalization on the result of the quadratic operation using Equation 8. In addition, the normalization operator 140 may activate and pool the quadratic result.

According to an embodiment of the present disclosure, the improved binarization method of the first layer of the convolutional neural network may be implemented in the form of program instructions that may be executed by various computer means and may be recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

The foregoing description of the application is intended for illustration, and it will be understood by those skilled in the art that the present invention may be easily modified in other specific forms without changing the technical spirit or essential features of the present application. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present application is indicated by the following claims rather than the above description, and it should be construed that all changes or modifications derived from the meaning and scope of the claims and their equivalents are included in the scope of the present application.

100: Improved Binarization Device of the First Layer of Convolutional Neural Networks
110: binarization parameter generator
120: convolution calculation unit
130: short cut operation
200: normalization operation unit

Claims (11)

An improved binarization method of a first layer of a convolutional neural network performed by an improved binarization apparatus of a first layer of a convolutional neural network,
(a) binarizing the initial input data for the first layer to generate binarization input data, and binarizing the parameter of the first layer to generate a binarization parameter;
(b) multiply the binarization parameter to generate a multiplication parameter, perform a convolution operation between the binarization input data and the multiplication parameter, and then perform a multiplication multiplication with the multiplying parameter on the convolution operation. Generating a first order operation result; And
(c) adding a short cut to the first operation result to calculate a second operation result,
And wherein said shortcut corresponds to said initial input data prior to binarization. The method of claim 1,
And by the addition operation of the shortcut in step (c), initial input data prior to binarization in step (a) is taken into account. The method of claim 1,
In step (b), the result of the first operation is batch normalized,
In step (c), wherein the shortcut is batch normalized. The method of claim 3,
The quadratic operation result is calculated by Equation 1 below,
[Equation 1]

Here, e _{1, m} is the coefficient of the m-th filter, (W ₁ ) ^b _{m, x, y, z} is a binarization parameter of the first layer, A - ₀ is the initial input data, the convolutional neural network Improved binarization of the first layer. The method of claim 1,
(d) further performing pooling and batch normalization on the result of the quadratic operation. An improved binarization device of the first layer of the convolutional neural network,
A binarization parameter generator for generating binarization input data by binarizing initial input data for the layer, and generating binarization parameters by binarizing the parameters of the first layer;
Performs multiplexing on the binarization parameter to generate a multiplexing parameter, performs a convolution operation between the binarization input data and the multiplexing parameter, and then performs a multiplication multiplication with the multiplexing parameter with respect to the convolution operation. A convolution operator for generating a difference operation result; And
Including a shortcut operation to calculate a secondary operation result by adding a shortcut to the first operation result,
And wherein said shortcut corresponds to said initial input data prior to binarization. The method of claim 6,
Wherein, by the addition operation of the shortcut, the initial input data before binarization is taken into account. The method of claim 6,
The convolution operation unit,
Batch normalize the result of the first operation,
The short cut operation unit,
Improved binarization of the first layer of the convolutional neural network, wherein the shortcut is batch normalized. The method of claim 8,
The short cut operation unit,
Equation 2 below to calculate the second operation result,
[Equation 2]

Here, e _{1, m} is the coefficient of the m th filter for the first layer, (W ₁ ) ^b _{m, x, y, z} is the binarization parameter of the first layer, A - ₀ is the initial input data Improved binarization of the first layer of the convolutional neural network. The method of claim 6,
And a normalization operator configured to perform pooling and batch normalization on the quadratic result. A computer-readable recording medium having recorded thereon a program for executing the method of any one of claims 1 to 5.

Images