KR102261715B1 - Method and system for bit quantization of artificial neural network - Google Patents

Abstract

The present disclosure provides a method for bit quantization of an artificial neural network. The method includes the steps of (a) selecting one parameter or one parameter group to be quantized in an artificial neural network; (b) a bit quantization step of reducing a data representation size of the selected parameter or parameter group in units of bits; (c) determining whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value; (d) repeating steps (a) to (c) when the accuracy of the artificial neural network is equal to or greater than the target value.

Description

Bit quantization method and system of artificial neural network {METHOD AND SYSTEM FOR BIT QUANTIZATION OF ARTIFICIAL NEURAL NETWORK}

The present disclosure relates to a method and system for bit quantization of an artificial neural network, and more particularly, to a method and system for bit quantization capable of reducing performance and memory usage while maintaining practical accuracy of an artificial neural network.

An artificial neural network is a computer structure that models a biological brain. In an artificial neural network, nodes corresponding to neurons in the brain are interconnected, and the strength of synaptic coupling between neurons is expressed as a weight. In the artificial neural network, artificial neurons (nodes) change the strength of synaptic bonds between nodes through learning, thereby constructing a model having a given problem-solving ability.

The artificial neural network, in a narrow sense, may refer to a multi-layered perceptron, which is a type of a feedforward neural network, but is not limited thereto, and a radial basis function network, a self-organized neural network. It may include various types of neural networks, such as a self-organizing network and a recurrent neural network.

Recently, a multi-layered deep neural network has been widely used as a technology for image recognition, and a representative example of a multi-layered deep neural network is a convolutional neural network (CNN). In the case of a general multi-layer forward transmission neural network, input data is limited to a one-dimensional form. When image data composed of two to three dimensions is flattened into one-dimensional data, spatial information is lost, and spatial information of the image is lost. It may be difficult to train a neural network in the state it is maintained. However, the convolutional neural network can learn about visual information while maintaining 2D or 3D spatial information.

Specifically, the convolutional neural network effectively recognizes features with adjacent images while maintaining spatial information of the image, and includes a Max Pooling process that collects and strengthens the features of the extracted image, so that it recognizes patterns in visual data. effective in However, a deep neural network with a multi-layer structure such as a convolutional neural network uses a deep layer structure to provide high recognition performance, but the structure is very complex and requires a large amount of computation and a large amount of memory. In multi-layer deep neural networks, most of the operations that occur internally are executed using multiplication and addition (or accumulation), where the number of connections between nodes in the neural network is large and parameters requiring multiplication (e.g., weight data, features Because the number of map data, activation map data, etc.) is large, a large amount of computation is required in the learning process or recognition process.

As described above, a large amount of computation and memory is required in the learning and recognition process of a deep neural network having a multi-layer structure such as a convolutional neural network. As a method of reducing the amount of computation and memory of a multi-layered deep neural network, a bit quantization method of reducing the data expression size of a parameter used for computation of an artificial neural network in bits can be used. In the conventional bit quantization method, uniform bit quantization, which quantizes all parameters of the artificial neural network with the same number of bits, is used, but the conventional uniform bit quantization method uses the number of bits for each parameter used in the artificial neural network. The problem is that the change does not accurately reflect the impact on overall performance.

The embodiments disclosed herein quantize each parameter data constituting an artificial neural network or parameter data grouped according to a specific criterion to a specific number of bits so as to maintain artificial intelligence accuracy while improving overall performance in the artificial neural network. It is intended to provide a method and system for doing so.

According to an embodiment of the present disclosure, a method for bit quantization of an artificial neural network is provided. The method includes the steps of: (a) selecting at least one parameter from among a plurality of parameters used in an artificial neural network; (b) a bit quantization step of reducing the size of data required for the operation on the selected parameter in units of bits; (c) determining whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value; (d) when the accuracy of the artificial neural network is equal to or greater than the target value, repeating steps (b) to (c) for the parameter to further reduce the number of bits in the data representation of the parameter have. Also, in this method, (e) when the accuracy of the artificial neural network is less than the target value, after restoring the number of bits of the parameter to the number of bits when the accuracy of the artificial neural network exceeds the target value, (a) It further comprises repeating steps to (d).

According to an embodiment of the present disclosure, a method for bit quantization of an artificial neural network is provided. The method includes the steps of: (a) selecting, by a parameter selection module, at least one of the plurality of layers; (b) a bit quantization step of reducing, by the bit quantization module, the size of the data representation for the parameter of the selected layer in bits; (c) determining, by the accuracy determination module, whether the accuracy of the artificial neural network is equal to or greater than a predetermined target value; and (d) repeating steps (a) to (c) when the accuracy of the artificial neural network is equal to or greater than the target value.

According to an embodiment of the present disclosure, a method for bit quantization of an artificial neural network is provided. The method includes the steps of: (a) selecting one or more data or one or more groups of data from weight, feature map, and activation map data in the artificial neural network by a parameter selection module; (b) a bit quantization step of reducing a data representation size of the selected data in bits by a bit quantization module; (c) measuring whether the artificial intelligence accuracy of the artificial neural network is greater than or equal to a target value; and (d) repeating steps (a) to (c) until there is no more data to be quantized among the data of the artificial neural network.

According to an embodiment of the present disclosure, a method for bit quantization of an artificial neural network is provided. The method includes: training the artificial neural network according to one or more parameters of the artificial neural network; performing bit quantization on one or more parameters of the artificial neural network according to the bit quantization method of the artificial neural network according to the embodiments; and training the artificial neural network according to one or more parameters of the artificial neural network on which the bit quantization is performed.

According to another embodiment of the present disclosure, a bit quantization system of an artificial neural network is provided. The system includes: a parameter selection module for selecting at least one parameter in the artificial neural network; a bit quantization step of reducing the size of the data representation of the selected parameter in bits; and an accuracy determination module for determining whether the accuracy of the artificial neural network is equal to or greater than a predetermined target value. When the accuracy of the artificial neural network is greater than or equal to the target value, the accuracy determination module controls the parameter selection module and the bit quantization module to maintain the accuracy of the artificial neural network to be greater than or equal to the target value, and each of the plurality of parameters is Quantization can be performed to have a minimum number of bits.

According to an embodiment of the present disclosure, a bit quantization system of an artificial neural network is provided. The system includes: a parameter selection module for selecting at least one layer from among a plurality of layers constituting the artificial neural network; a bit quantization module for reducing the size of the data representation for the parameter of the selected layer in units of bits; and an accuracy determination module for determining whether the accuracy of the artificial neural network is equal to or greater than a predetermined target value, wherein the accuracy determination module is configured to, when the accuracy of the artificial neural network is greater than or equal to the target value, the parameter selection module and the bit quantization control the module to perform bit quantization on another one of the plurality of layers, and the bit quantization module is configured to: n bits for all weights of the plurality of layers (where n is an integer where n > 0) ), and set m bits (where m is an integer where m>0) for the output data of the plurality of layers.

According to an embodiment of the present disclosure, a bit quantization system of an artificial neural network is provided. The system includes: a parameter selection module for selecting at least one layer from among a plurality of layers constituting the artificial neural network; a bit quantization module for reducing the size of the data representation for the parameter of the selected layer in units of bits; and an accuracy determination module for determining whether the accuracy of the artificial neural network is equal to or greater than a predetermined target value, wherein the accuracy determination module is configured to, when the accuracy of the artificial neural network is greater than or equal to the target value, the parameter selection module and the bit quantization The module is controlled to perform bit quantization on another one of the plurality of layers, and the bit quantization module is configured for n bits (where n is n > 0) for weights and output data of the plurality of layers. integer), but the number of bits allocated to each of the plurality of layers is set differently.

According to an embodiment of the present disclosure, a bit quantization system of an artificial neural network is provided. The system includes: a parameter selection module for selecting at least one layer from among a plurality of layers constituting the artificial neural network; a bit quantization module for reducing the size of the data representation for the parameter of the selected layer in units of bits; and an accuracy determination module for determining whether the accuracy of the artificial neural network is equal to or greater than a predetermined target value, wherein the accuracy determination module is configured to, when the accuracy of the artificial neural network is greater than or equal to the target value, the parameter selection module and the bit quantization control the module to perform bit quantization for another one of the plurality of layers, wherein the bit quantization module allocates weights of the plurality of layers and the number of bits of output data differently.

According to an embodiment of the present disclosure, there is provided a bit quantization system of an artificial neural network. The system includes: a parameter selection module for selecting at least one layer from among a plurality of layers constituting the artificial neural network; a bit quantization module for reducing the size of a memory for storing the parameter of the selected layer in units of bits; and an accuracy determination module for determining whether the accuracy of the artificial neural network is equal to or greater than a predetermined target value, wherein the accuracy determination module is configured to, when the accuracy of the artificial neural network is greater than or equal to the target value, the parameter selection module and the bit quantization The module is controlled to perform bit quantization on another one of the plurality of layers, and the bit quantization module allocates a different number of bits for each weight used in the plurality of layers.

According to an embodiment of the present disclosure, a bit quantization system of an artificial neural network is provided. The system includes: a parameter selection module for selecting at least one layer from among a plurality of layers constituting the artificial neural network; a bit quantization module for reducing the size of the data representation for the parameter of the selected layer in units of bits; and an accuracy determination module for determining whether the accuracy of the artificial neural network is equal to or greater than a predetermined target value, wherein the accuracy determination module is configured to, when the accuracy of the artificial neural network is greater than or equal to the target value, the parameter selection module and the bit quantization control the module to perform bit quantization for another one of the plurality of layers, wherein the bit quantization module allocates a different number of bits individually to a specific unit of output data output from the plurality of layers do.

According to an embodiment of the present disclosure, a bit quantization system of an artificial neural network is provided. The system includes: a parameter selection module for selecting at least one layer from among a plurality of layers constituting the artificial neural network; a bit quantization module for reducing the size of the data representation for the parameter of the selected layer in units of bits; and an accuracy determination module for determining whether the accuracy of the artificial neural network is equal to or greater than a predetermined target value, wherein the accuracy determination module is configured to, when the accuracy of the artificial neural network is greater than or equal to the target value, the parameter selection module and the bit quantization The module is controlled to perform bit quantization on another one of the plurality of layers, and the bit quantization module allocates different bits to individual values of output data output from the plurality of layers.

According to various embodiments of the present disclosure, overall computational performance may be improved by quantizing the number of bits of data required for computation, such as learning or inference, in an artificial neural network. In addition, it is possible to implement an artificial neural network without deterioration of artificial intelligence accuracy while reducing hardware resources required to implement the artificial neural network, reducing power consumption and memory required usage.

The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the range of cheongguk.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, wherein like reference numerals denote like elements, but are not limited thereto.
1 is a diagram illustrating an example of an artificial neural network for obtaining output data for input data using a plurality of layers and a plurality of layer weights according to an embodiment of the present disclosure.
2 to 3 are diagrams for explaining specific implementations of the artificial neural network shown in FIG. 1 according to an embodiment of the present disclosure.
4 is a diagram illustrating another example of an artificial neural network including a plurality of layers according to an embodiment of the present disclosure.
5 is a diagram illustrating a weight kernel used in a convolution operation with input data in a convolution layer according to an embodiment of the present disclosure.
6 is a diagram illustrating a procedure of generating a first activation map by performing convolution on input data using a first weight kernel according to an embodiment of the present disclosure.
7 is a diagram illustrating a procedure of generating a second activation map by performing convolution on input data using a second weight kernel according to an embodiment of the present disclosure.
8 is a diagram illustrating an operation process of a convolutional layer in a matrix according to an embodiment of the present disclosure.
9 is a diagram illustrating a matrix operation of a fully connected layer according to an embodiment of the present disclosure.
10 is a diagram illustrating a process of bit quantization of a convolutional layer in a matrix according to an embodiment of the present disclosure.
11 is a flowchart illustrating a bit quantization method of an artificial neural network according to an embodiment of the present disclosure.
12 is a flowchart illustrating a bit quantization method of an artificial neural network according to another embodiment of the present disclosure.
13 is a flowchart illustrating a bit quantization method of an artificial neural network according to another embodiment of the present disclosure.
14 is a graph illustrating an example of an amount of computation for each layer of an artificial neural network according to an embodiment of the present disclosure.
15 is a graph illustrating the number of bits per layer of an artificial neural network in which bit quantization is performed by a forward bit quantization method according to an embodiment of the present disclosure.
16 is a graph illustrating the number of bits per layer of an artificial neural network in which bit quantization is performed by a backward bit quantization method according to an embodiment of the present disclosure.
17 is a graph illustrating the number of bits per layer of an artificial neural network in which bit quantization is performed by a high computational cost layer first bit quantization method according to an embodiment of the present disclosure.
18 is a graph showing the number of bits per layer of an artificial neural network in which bit quantization is performed by a low computational cost layer first bit quantization method according to an embodiment of the present disclosure.
19 is a diagram illustrating a hardware implementation example of an artificial neural network according to an embodiment of the present disclosure.
20 is a diagram illustrating an example of hardware implementation of an artificial neural network according to another embodiment of the present disclosure.
21 is a diagram illustrating an example of hardware implementation of an artificial neural network according to another embodiment of the present disclosure.
22 is a diagram illustrating a configuration of a system for performing bit quantization on an artificial neural network according to an embodiment of the present disclosure.

Hereinafter, specific contents for carrying out the present disclosure will be described in detail with reference to the accompanying drawings. However, in the following description, if there is a risk of unnecessarily obscuring the subject matter of the present disclosure, detailed descriptions of well-known functions or configurations will be omitted.

In the accompanying drawings, identical or corresponding components are assigned the same reference numerals. In addition, in the description of the embodiments below, overlapping description of the same or corresponding components may be omitted. However, even if descriptions regarding components are omitted, it is not intended that such components are not included in any embodiment.

In the present disclosure, a “parameter” may refer to an artificial neural network or any one or more of weight data, feature map data, and activation map data of each layer constituting the artificial neural network. In addition, "parameter" may mean an artificial neural network represented by such data or each layer constituting an artificial neural network. Also, in the present disclosure, “bit quantization” may refer to an operation or operation that reduces the number of bits in a data representation representing a parameter or group of parameters.

The present disclosure provides various embodiments of a quantization method and system for reducing the data representation size of a parameter used for a related operation in bits in order to reduce the amount of computation, memory usage, and power consumption of a digital hardware system. In some embodiments, the bit quantization method and system of the present disclosure may reduce the size of a parameter used for calculation of an artificial neural network in bits. In general, a 32-bit, 16-bit, or 8-bit data structure (eg, CPU, GPU, memory, cache, buffer, etc.) is used for the operation of an artificial neural network. Accordingly, the quantization method and system of the present disclosure can reduce the size of a parameter used for calculation of an artificial neural network to bits other than 32, 16, and 8 bits. Moreover, a specific number of bits can be individually and differently assigned to each parameter or group of each parameter of the artificial neural network.

In some embodiments, the bit quantization method and system of the present disclosure set n bits (n is an integer such that n > 0) for all weights for an artificial neural network model, and convert the output data of each layer to m bits (m is an integer where m > 0) can be set.

In another embodiment, the bit quantization method and system of the present disclosure may allocate n bits to the weight and output data of each layer of the artificial neural network model, where n may be set to a different number for each layer.

In another embodiment, the bit quantization method and system of the present disclosure allocate different bits to the weights and output data of each layer of the artificial neural network model, and also for the weights for each layer and the output feature map parameters in the corresponding layer. A different number of bits can be allocated.

The bit quantization method and system of the present disclosure can be applied to various types of artificial neural networks. For example, when the bit quantization method and system of the present disclosure are applied to a convolutional neural network (CNN), different bits may be individually allocated to weight kernels used in each layer of the artificial neural network. can

In another embodiment, the bit quantization method and system of the present disclosure allocate different bits to each weight used in each layer of a multi-layered artificial neural network model, or separate bits in a specific unit of output data of each layer. Alternatively, different bits may be allocated to individual values of the output data of each layer.

In the bit quantization method and system according to various embodiments of the present disclosure described above, any one of the above-described embodiments may be applied to an artificial neural network model, but is not limited thereto, and one or more of these embodiments may be applied to an artificial neural network model. It can also be applied to artificial neural network models by combining them.

1 is a diagram illustrating an example of an artificial neural network 100 for obtaining output data for input data using a plurality of layers and a plurality of layer weights according to an embodiment of the present disclosure.

In general, a multi-layered artificial neural network such as the artificial neural network 100 is a statistical learning algorithm implemented based on the structure of a biological neural network or a structure for executing the algorithm in machine learning technology and cognitive science. include That is, the artificial neural network 100 repeatedly adjusts the weight of the synapse by repeatedly adjusting the weight of the synapse in the artificial neural network 100, which is an artificial neuron that forms a network by combining synapses, as in a biological neural network, to obtain correct output and inferred output corresponding to a specific input. By learning to reduce the error between them, it is possible to create a machine learning model with problem-solving ability.

In one example, the artificial neural network 100 may be implemented as a multilayer perceptron (MLP) consisting of layers including one or more nodes and a connection therebetween. However, the artificial neural network 100 according to the present embodiment is not limited to the structure of the MLP, and may be implemented using one of various artificial neural network structures having a multi-layered structure.

As shown in FIG. 1 , when input data is input from the outside, the artificial neural network 100 corresponds to the input data through a plurality of layers 110_1 , 110_2 , ..., 110_N each composed of one or more nodes. configured to output output data.

In general, the learning method of the artificial neural network 100 includes a supervised learning method that learns to be optimized to solve a problem by input of a teacher signal (correct answer), and an unsupervised learning method that does not require a teacher signal. Learning) method, there is a semi-supervised learning method that uses both supervised learning and unsupervised learning. The artificial neural network 100 shown in FIG. 1 performs at least one of a supervised learning method, an unsupervised learning method, and a semi-supervised learning method according to a user's selection. By using it, the artificial neural network 100 that generates output data can be trained.

2 to 3 are diagrams for explaining specific implementations of the artificial neural network 100 shown in FIG. 1 according to an embodiment of the present disclosure.

,

...

,

), 입력 데이터(210)에 대응하는 출력 데이터를 출력하는 출력 노드(

,

...

,

), 입력 노드와 출력 노드 사이에 위치하는 은닉 노드 및 다수의 파라미터를 포함할 수 있다. 입력 노드(

,

...

,

)는, 입력층(220)을 구성하는 노드로서, 외부로부터 입력 데이터(210)(예를 들어, 이미지)를 수신하고, 출력 노드(

,

...

,

)는 출력층(240)을 구성하는 노드로서, 외부로 출력데이터를 출력할 수 있다. 입력 노드와 출력 노드 사이에 위치한 은닉 노드는, 은닉층(230)을 구성하는 노드로서, 입력 노드의 출력 데이터를 출력 노드의 입력 데이터로 연결할 수 있다. 입력층(220)의 각 노드는, 도 2에 도시된 바와 같이, 출력층(240)의 각 출력 노드와 완전 연결될 수 있고, 불완전 연결될 수 있다. 또한, 입력 노드는, 외부로부터 입력 데이터를 수신하여 은닉 노드로 전달해주는 역할을 할 수 있다. 이때, 은닉 노드와 출력 노드에서는, 데이터에 대한 계산을 수행할 수 있는데, 수신한 입력 데이터에 파라미터(또는 가중치)를 곱하여 계산을 수행할 수 있다. 각 노드의 계산이 완료되면, 계산 결과값을 모두 합한 후, 미리 설정된 활성화 함수를 이용하여 출력 데이터를 출력할 수 있다. Referring to FIG. 2 , the artificial neural network 200 is an input node (

,

...

,

), an output node that outputs output data corresponding to the input data 210 (

,

...

,

), a hidden node located between the input node and the output node, and a number of parameters. input node (

,

...

,

) is a node constituting the input layer 220, which receives input data 210 (eg, an image) from the outside, and an output node (

,

...

,

) is a node constituting the output layer 240 , and may output output data to the outside. A hidden node located between the input node and the output node is a node constituting the hidden layer 230 and may connect output data of the input node to the input data of the output node. Each node of the input layer 220 may be fully connected or incompletely connected to each output node of the output layer 240 as shown in FIG. 2 . Also, the input node may serve to receive input data from the outside and deliver it to the hidden node. In this case, the hidden node and the output node may perform calculation on data, and may perform calculation by multiplying the received input data by a parameter (or weight). When the calculation of each node is completed, the output data may be output by using a preset activation function after summing all the calculation results.

,

...

,

)는 활성화 함수를 갖는다. 활성화 함수는 계단 함수(step function), 부호 함수(sign function), 선형 함수(linear function), 로지스틱 시그모이드 함수(logistic sigmoid function), 하이퍼탄젠트 함수(hyper tangent function), ReLU 함수, 소프트맥스(softmax) 함수 중 어느 하나일 수 있다. 활성화 함수는 통상의 기술자라면 인공 신경망의 학습 방법에 따라 적절히 결정될 수 있다. Hidden nodes and output nodes (

,

...

,

) has an activation function. Activation functions are step function, sign function, linear function, logistic sigmoid function, hyper tangent function, ReLU function, softmax ( softmax) function. An activation function may be appropriately determined by a person skilled in the art according to a learning method of an artificial neural network.

The artificial neural network 200 performs machine learning by repeatedly updating (or correcting) weight values to appropriate values. As a method for the artificial neural network 200 to perform machine learning, there are representatively supervised learning and unsupervised learning.

Supervised learning updates the weight values so that the output data obtained by putting the input data into the neural network becomes similar to the target data in a state in which the target output data desired to be calculated by an arbitrary neural network is clearly defined for the input data. learning method. The multi-layered artificial neural network 200 of FIG. 2 may be generated based on supervised learning.

Referring to FIG. 3 , as another example of an artificial neural network having a multilayer structure, there is a convolutional neural network (CNN) 300 which is a type of a deep neural network (DNN). A convolutional neural network (CNN) is a neural network composed of one or several convolutional layers, a pooling layer, and fully connected layers. A convolutional neural network (CNN) has a structure suitable for learning two-dimensional data, and can be learned through a backpropagation algorithm. It is one of the representative models of DNN that is widely used in various application fields such as object classification in images and object detection.

Here, it should be noted that the multi-layered artificial neural network of the present invention is not limited to the artificial neural network shown in FIGS. 2 and 3, and a learned model may be obtained by machine learning other types of data in various other artificial neural networks. do.

4 is a diagram illustrating another example of an artificial neural network including a plurality of layers according to an embodiment of the present disclosure. The artificial neural network 400 shown in FIG. 1 is a plurality of convolution layers (CONV) 420, a plurality of subsampling layers (SUBS) 430, and a plurality of convolution layers (CONV) 420 shown in FIG. It is a convolutional neural network (CNN) including a fully-connected layer (FC) 440 of

The CONV 420 of the CNN 400 generates a feature map by applying a convolution weight kernel to the input data 410 . Here, the CONV 420 may serve as a kind of template for extracting features from high-dimensional input data (eg, image or video). Specifically, one convolution may be repeatedly applied several times while changing the location of a portion of the input data 410 to extract features from the entire input data 410 . In addition, the SUBS 430 serves to reduce the spatial resolution of the feature map generated by the CONV 420 . The subsampling functions to reduce the dimension of the input data (eg, a feature map), thereby reducing the complexity of the analysis problem of the input data 410 . The SUBS 430 may use a max pooling operator that takes a maximum value or an average pooling operator that takes an average value with respect to values of a part of the feature map. The SUBS 430 has the effect of not only reducing the dimension of the feature map through a pooling operation, but also making the feature map robust against shift and distortion. Finally, the FC 440 may perform a function of classifying input data based on the feature map.

CNN 400 may execute various configurations and functions according to the number of layers of CONV 420 , SUBS 430 , and FC 440 or types of operators. For example, the CNN 400 may include any one of various CNN configurations such as AlexNet, VGGNet, LeNet, ResNet, and the like, but is not limited thereto.

The CONV 420 of the CNN 400 having the above-described configuration can generate a feature map through convolution operation by applying a weight to the input data 410 when the image data is input as the input data 410. , in this case, a group of weights used may be referred to as a weight kernel. The weight kernel is a three-dimensional matrix of nxmxd, where n represents a row of a specific size as in the input image data, m represents a column of a specific size, d represents a channel of the input image data, etc. number is an integer greater than or equal to 1), and it is possible to generate a feature map through convolution operation by traversing the input data 410 at a specified interval. At this time, if the input data 410 is a color image having a plurality of channels (eg, three channels of RGB), the weight kernel traverses each channel of the input data 410 and calculates a convolution, You can create a feature map.

5 is a diagram illustrating a weight kernel used for a convolution operation with input data of a convolution layer according to an embodiment of the present disclosure.

As illustrated, the input data 510 may be an image or an image displayed as a two-dimensional matrix composed of a row 530 of a specific size and a column 540 of a specific size. As described above, the input data 510 may have a plurality of channels 550 , where the channels 550 may represent the number of color components of the input data image. Meanwhile, the weight kernel 520 may be a weight kernel used for convolution for extracting a feature of a portion of the input data 510 while scanning it. The weight kernel 520 may be configured to have a specific size of a row 560 , a specific size of a column 570 , and a specific number of channels 580 similarly to the input data image. In general, the size of the row 560 and the column 570 of the weight kernel 520 is set to be the same, and the number of channels 580 may be the same as the number of channels 550 of the input data image.

6 is a diagram for describing a procedure of generating a first activation map by performing convolution on input data using a first kernel according to an embodiment of the present disclosure.

The first weight kernel 610 may be a weight kernel indicating the first channel of the weight kernel 620 of FIG. 2 . The first weight kernel 610 may finally generate the first activation map 630 by traversing the input data at a specified interval and performing convolution. Convolution, when the first weight kernel 610 is applied to a part of the input data 510, multiplies the input data values at a specific position of the part and the values at the corresponding position of the weight kernel, and then uses all of the values generated. is executed in addition. Through this convolution process, a first result value 620 is generated, and whenever the first weight kernel 610 traverses the input data 510 , the result values of the convolution are generated to configure a feature map. . Each component value of the feature map is converted into the first activation map 630 through the activation function of the convolutional layer.

7 is a diagram illustrating a procedure of generating a second activation map by performing convolution on input data using a second weight kernel according to an embodiment of the present disclosure.

As shown in FIG. 6 , the first activation map 620 is generated by performing convolution on the input data 510 using the first weight kernel 610 , and then, as shown in FIG. 7 , the second weight The second activation map 730 may be generated by performing convolution on the input data 510 using the kernel 710 .

The second weight kernel 710 may be a weight kernel indicating the second channel of the weight kernel 520 of FIG. 5 . The second weight kernel 710 may finally generate the second activation map 730 by traversing the input data at specified intervals and performing convolution. As in FIG. 6 , the convolution, when the second weight kernel 710 is applied to a portion of the input data 510 , multiplies the input data values of a specific position of the portion and the values of the corresponding position of the weight kernel, respectively. It is executed by adding all the generated values. Through this convolution process, a second result value 720 is generated, and whenever the second weight kernel 710 traverses the input data 510 , the result value of the convolution is generated to configure a feature map. . Each component value of the feature map is converted into the second activation map 730 through the activation function of the convolutional layer.

8 is a diagram illustrating a process of calculating a convolutional layer in a matrix when an input feature map has one channel according to an embodiment of the present disclosure.

The convolutional layer 420 illustrated in FIG. 8 may correspond to the CONV 420 illustrated in FIG. 4 . In FIG. 8 , input data 810 input to the convolutional layer 420 is represented as a two-dimensional matrix having a size of 6×6, and the weight kernel 814 is a two-dimensional matrix having a size of 3×3. is displayed However, the size of the input data 810 and the weight kernel 814 of the convolution layer 420 is not limited thereto, and the size of the input data 810 and the weight kernel 814 of the convolution layer 420 may vary according to the performance and requirements of the artificial neural network including the convolution layer 420 . can be changed.

As shown, when the input data 810 is input to the convolutional layer 420, the weight kernel 814 traverses the input data 810 at a predetermined interval (eg, 1), and the input data ( Elementwise multiplication may be performed by multiplying the values of the 810 and the weight kernel 814 at the same location, respectively. The weight kernel 814 traverses the input data 810 at regular intervals and sums values obtained through multiple multiplication (summation 816).

Specifically, the weight kernel 814 assigns a value (eg, "3") of multiple products calculated at a specific location 820 of the input data 810 to the corresponding element 824 of the feature map 818 . do. Next, the weight kernel 814 assigns the multiple product value (eg, “1”) calculated at the next position 822 of the input data 810 to the corresponding element 826 of the feature map 818 . do. In this way, when all values of multiple products calculated by the weight kernel 814 while traversing the input data 810 are allotted to the feature map 818, the feature map 818 having a size of 4 x 4 is completed. At this time, if the input data 810 is composed of, for example, three channels (R channel, G channel, and B channel), the same weighted kernel or different channels for each channel are applied to the data phase for each channel of the input data 810 , respectively. It is possible to generate feature maps for each channel through convolution in which multiple products 812 and 816 are traversed.

Referring back to FIG. 4 , the CONV 420 applies an activation function to the feature map generated according to the method described with reference to FIGS. 25 to 8 to obtain an activation map that is a final output result of the convolutional layer. can create Here, the activation function is any one of various activation functions, such as a sigmoid function, a radial basis function (RBF), and a rectified linear unit (ReLU), or a modified function of them. or it can be another function.

Meanwhile, the SUBS 430 receives the activation map, which is output data of the CONV 420 , as input data. The SUBS 430 performs a function of reducing the size of the activation map or emphasizing specific data. When the SUBS 430 uses max pooling, the maximum value of a value in a specific region of the activation map is selected and output. As described above, noise of input data can be removed through the pooling process of the SUBS 430 and the size of the data can be reduced.

Also, the FC 440 may receive the output data of the SUBS 430 to generate the final output data 450 . The activation map extracted from the SUBS 430 is one-dimensionally flattened to be input to the fully connected layer 440 .

9 is a diagram illustrating a matrix operation of a fully connected layer according to an embodiment of the present disclosure.

The fully connected layer 440 illustrated in FIG. 9 may correspond to the FC 440 of FIG. 4 . As described above, the activation map extracted from the max pooling layer 430 may be flattened in one dimension to be input to the fully connected layer 440 . The one-dimensional smoothed activation map may be received as input data 910 in the fully connected layer 440 . In the fully connected layer 440 , multiple products 912 of the input data 910 and the weight kernel 914 may be executed using the one-dimensional weight kernel 914 . A result value of the multiple product of the input data 910 and the weight kernel 914 may be summed 916 and output as the output data 918 . In this case, the output data 918 may represent an inference value for the input data 410 input to the CNN 400 .

The CNN 400 having the configuration described above receives input data of a two-dimensional or one-dimensional matrix for each of a plurality of layers, and learns and infers through complex operations such as multiple multiplication and summation of a weighted kernel for the input data. run the process Accordingly, resources (eg, the number of operators or the amount of memory) required for data learning and inference may be significantly increased according to the number of layers constituting the CNN 400 or the complexity of operations. Accordingly, bit quantization may be performed on input/output data used for each layer in order to reduce the amount of computation and memory of an artificial neural network having a plurality of layers, such as the CNN 400 . In an embodiment, the bit quantization of the CNN 400 having a plurality of layers may be performed for the CONV 420 and the FC 440 requiring a large amount of computation and memory.

10 is a diagram illustrating a process of bit quantization of a convolutional layer in a matrix according to an embodiment of the present disclosure.

The bit quantization performed in the convolutional layer is a weight or weight kernel quantization 1028 that reduces the number of bits of each element value of the weight kernel used for the convolution operation, and/or the value of each element of the feature map or activation map. It may include feature map quantization or activation map quantization 1030 that reduces the number of bits.

The bit quantization process of the convolutional layer according to an embodiment may be performed as follows. Before convolution is performed by applying the weight kernel 1014 to the input data 1010 of the convolutional layer, a quantization 716 process is performed on the weight kernel 1014 to generate a quantized weight kernel 1018 . In addition, the quantized weight kernel 1018 is applied to the input data 1010, multiple products 1012 and summation 1020 are executed to output the values of the convolution to generate a feature map, and then activate it through an activation function. A map 1022 may be generated. Next, a final quantized activation map 1026 may be generated through quantization 1024 on the activation map.

In the bit quantization process of the convolutional layer described above, the weight kernel quantization 1028 may be performed using the following equation.

는 양자화될 가중치 값(예를 들어, 실수의 가중치 및 가중치 커널 내의 각 가중치)을 나타내고,

는 양자화할 비트 수를 나타내고,

는

가k비트 만큼 양자화된 결과를 나타낸다. 즉, 위 수식에 따르면, 먼저

에 대해, 사전 결정된 이진수

를 곱하여 ,

가 k 비트만큼 자리수가 증가된다(이하 "제1 값"이라고 함). 다음으로, 제1 값에 대해 라운딩(rounding) 또는 트렁케이션(truncation) 연산을 실행함으로써,

의 소수점 이하 숫자가 제거된다(이하 "제2 값"이라고 함). 제2 값은 이진수

으로 나누어, k 비트만큼 자리수가 다시 감소됨으로써, 최종 양자화된 가중치 커널의 요소 값이 계산될 수 있다. 이와 같은 가중치 또는 가중치 커널 양자화(1028)는 가중치 또는 가중치 커널(1014)의 모든 요소 값에 대해 반복 실행되어, 양자화 된 가중치 값들(1018)이 생성된다.here,

denotes the weight values to be quantized (eg, real weights and each weight in the weight kernel),

represents the number of bits to be quantized,

is

It represents the result quantized by k bits. That is, according to the above formula, first

For , a predetermined binary number

Multiply by ,

is incremented by k bits (hereinafter referred to as "first value"). Next, by executing a rounding or truncation operation on the first value,

The number after the decimal point of ' is removed (hereinafter referred to as "second value"). the second value is binary

By dividing by , the number of digits is again reduced by k bits, so that the element value of the final quantized weight kernel can be calculated. This weight or weight kernel quantization 1028 is iteratively executed for all element values of the weight or weight kernel 1014 , thereby generating quantized weight values 1018 .

Meanwhile, the feature map or activation map quantization 1030 may be performed by the following equation.

)(예를 들어, 실수의 계수)에 대한 양자화가 적용되기 전에, 클립핑(clipping)이 적용하여 특징맵 또는 활성화 맵(1022)의 각 요소 값을 0에서 1의 사이 값으로 정규화 시키는 과정을 추가할 수 있다. In the feature map or activation map quantization 1030, the same formula as the weight or weight kernel quantization 1028 method may be used. However, in the feature map or activation map quantization 1030, each element value (

) (e.g., coefficients of real numbers) before quantization is applied, clipping is applied to normalize each element value of the feature map or activation map 1022 to a value between 0 and 1 is added. can do.

에 대해, 사전 결정된 이진수

를 곱하여,

가 k 비트만큼 자리수가 증가된다("제1 값"). 다음으로, 제1 값에 대해 라운딩 또는 트렁케이션 연산을 실행함으로써,

의 소수점 이하 숫자가 제거된다("제2 값"). 제2 값은 이진수

으로 나누어, k 비트만큼 자리수가 다시 감소됨으로써, 최종 양자화된 특징맵 또는 활성화 맵(1026)의 요소 값이 계산될 수 있다. 이와 같은 특징맵 또는 활성화 맵의 양자화(1030)는 특징맵 또는 활성화 맵(1022)의 모든 요소 값에 대해 반복 실행되어, 양자화된 특징맵 또는 활성화 맵(1026)이 생성된다.Next, normalized

For , a predetermined binary number

Multiply by

is incremented by k bits (“first value”). Next, by executing a rounding or truncation operation on the first value,

's decimal digits are removed ("second value"). the second value is binary

By dividing by , the number of digits is again reduced by k bits, so that the element value of the final quantized feature map or activation map 1026 can be calculated. The quantization 1030 of the feature map or activation map is iteratively executed for all element values of the feature map or activation map 1022 , thereby generating a quantized feature map or activation map 1026 .

Through the above-described weight or weight kernel quantization 1028 and feature map or activation map quantization 1030, the memory size and amount of computation required for the convolution operation of the convolutional layer 420 of the convolutional neural network can be reduced in bits. can

11 is a flowchart illustrating a bit quantization method of an artificial neural network according to an embodiment of the present disclosure. This embodiment is an example in which the unit of a data group that can be quantized in the artificial neural network is assumed to be all parameters belonging to each layer constituting the artificial neural network.

As shown, the bit quantization method 1100 of the artificial neural network may start with the step of selecting at least one layer from among a plurality of layers included in the artificial neural network ( S1110 ). Which layer to select from among a plurality of layers included in the artificial neural network may be determined according to the effect of the selected layer on the overall performance or computational amount (or memory amount) of the artificial neural network. In an embodiment, in the multi-layered artificial neural network described above with reference to FIGS. 1 to 3 , a layer having a large influence on overall performance or computational amount of the artificial neural network may be arbitrarily selected. In addition, in the case of the convolutional artificial neural network (CNN) 400 described with reference to FIGS. 4 to 10 , the convolutional layer 420 and/or the fully connected layer 440 is the overall performance or amount of computation of the CNN 400 . At least one of the layers 420 and 440 may be selected because the effect on the etc. is large.

The method of selecting at least one of the plurality of layers included in the artificial neural network may be determined according to the effect of the selected layer on the overall performance or the amount of computation of the artificial neural network, but is not limited thereto, and may include one of various methods. can For example, the selection of at least one layer among a plurality of layers included in the artificial neural network is sequentially performed from a first layer from which input data is received to a subsequent layer according to the arrangement order of the plurality of layers constituting the artificial neural network. (ii) a method of sequentially selecting from the last layer in which the final output data is generated to the previous layer according to the arrangement order of the plurality of layers constituting the artificial neural network, (iii) the plurality of layers constituting the artificial neural network. It may be performed according to a method of selecting a layer with the highest amount of computation from among the layers, or (iv) a method of selecting a layer with the least amount of computation from among a plurality of layers constituting the artificial neural network.

When the selection of the layer of the artificial neural network is completed in step S1110, the step of reducing the data representation size of the parameter (eg, weight) of the selected layer in bits (S1120) may be performed.

In an embodiment, when the weight or the size of output data among the parameters of the selected layer is reduced in bits, the weight kernel quantization 1028 and the activation map quantization 1024 described with reference to FIGS. 4 to 10 may be performed. have. For example, the weight kernel quantization 1028 may be calculated by the following equation.

는 양자화될 가중치 커널의 요소 값(예를 들어, 실수의 가중치 커널 계수)을 나타내고,

는 양자화할 비트 수를 나타내고,

는

가k비트 만큼 양자화된 결과를 나타낸다. 즉, 위 수식에 따르면, 먼저

에 대해, 사전 결정된 이진수

를 곱하여 ,

가 k 비트만큼 자리수가 증가된다("제1 값"). 다음으로, 제1 값에 대해 라운딩 또는 트렁케이션 연산을 실행함으로써,

의 소수점 이하 숫자가 제거된다("제2 값"). 제2 값은 이진수

으로 나누어, k 비트만큼 자리수가 다시 감소됨으로써, 최종 양자화된 가중치 커널의 요소 값이 계산될 수 있다. 이와 같은 가중치 커널 양자화(1028)는 가중치 커널(1014)의 모든 요소 값에 대해 반복 실행되어, 양자화 가중치 커널(1018)이 생성된다.here,

denotes the element values of the weight kernel to be quantized (eg, real weight kernel coefficients),

represents the number of bits to be quantized,

is

It represents the result quantized by k bits. That is, according to the above formula, first

For , a predetermined binary number

Multiply by ,

is incremented by k bits (“first value”). Next, by executing a rounding or truncation operation on the first value,

's decimal digits are removed ("second value"). the second value is binary

By dividing by , the number of digits is again reduced by k bits, so that the element value of the final quantized weight kernel can be calculated. This weight kernel quantization 1028 is iteratively executed for all element values of the weight kernel 1014 to generate a quantized weight kernel 1018 .

Meanwhile, the activation map quantization 1030 may be performed by the following equation.

)(예를 들어, 실수의 계수)에 대한 양자화가 적용되기 전에, 클립핑(clipping)이 적용하여 활성화 맵(1022)의 각 요소 값을 0에서 1의 사이 값으로 정규화 시키는 과정을 추가할 수 있다. 다음으로, 정규화된

에 대해, 사전 결정된 이진수

를 곱하여,

가 k 비트만큼 자리수가 증가된다("제1 값"). 다음으로, 제1 값에 대해 라운딩 또는 트렁케이션 연산을 실행함으로써,

의 소수점 이하 숫자가 제거된다("제2 값"). 제2 값은 이진수

으로 나누어, k 비트만큼 자리수가 다시 감소됨으로써, 최종 양자화된 활성화 맵(1026)의 요소 값이 계산될 수 있다. 이와 같은 활성화 맵의 양자화(1030)는 활성화 맵(1022)의 모든 요소 값에 대해 반복 실행되어, 양자화 활성화 맵(1026)이 생성된다.In the activation map quantization 1030, each element value of the activation map 1022 (

) (e.g., coefficients of real numbers) before quantization is applied, a process of normalizing each element value of the activation map 1022 to a value between 0 and 1 by applying clipping may be added. . Next, normalized

For , a predetermined binary number

Multiply by

is incremented by k bits (“first value”). Next, by executing a rounding or truncation operation on the first value,

's decimal digits are removed ("second value"). the second value is binary

By dividing the number by k bits again, the element values of the final quantized activation map 1026 can be calculated. The quantization 1030 of the activation map as described above is iteratively executed for all element values of the activation map 1022 , thereby generating the quantization activation map 1026 .

In the above-described embodiments, an example of reducing the weight value or the number of bits of the activation map data has been described in order to reduce the size of the data expression for the parameter of the layer selected in the artificial neural network, but the bit quantization method of the present disclosure is However, the present invention is not limited thereto. In another embodiment, different bits may be allocated to data in the interruption stage existing between several operation stages for various data included in a layer selected in the artificial neural network. Accordingly, in order to reduce the size of the memory (eg, buffer, register, or cache) in which each data is stored when implemented as hardware of the artificial neural network, the number of bits of each data stored in the corresponding memory is reduced and the number of bits of the corresponding memory is reduced. The number of bits may be reduced. In another embodiment, the size of data bits of a data path through which data of a layer selected in the artificial neural network is transmitted may be reduced in units of bits.

After the execution of step S1120, a step S1130 of determining whether the accuracy of the artificial neural network is equal to or greater than a predetermined target value may proceed. After reducing the data expression size of the parameter of the selected layer in the artificial neural network in bits, if the accuracy of the output result of the corresponding artificial neural network (for example, the learning result or inference result of the artificial neural network) is greater than or equal to the predetermined target value, additionally It can be expected that the overall performance of the artificial neural network can be maintained even if the bits of the corresponding data are reduced.

Accordingly, when it is determined in step S1130 that the accuracy of the artificial neural network is equal to or greater than the target value, the process proceeds to step S1120 to further reduce the data representation size of the selected layer in bits. In addition, it may be determined again whether the accuracy of the output result of the artificial neural network (eg, the learning result or inference result of the artificial neural network) is equal to or greater than a predetermined target value (step S1130 ).

In step S1130, if the accuracy of the artificial neural network is not equal to or greater than the target value, it may be determined that the accuracy of the artificial neural network is degraded due to the currently executed bit quantization. Accordingly, in this case, the minimum number of bits that satisfy the accuracy target value in the bit quantization performed immediately before may be determined as the final number of bits for the parameter of the selected layer (step S1140 ).

Next, it is determined whether bit quantization for all layers of the artificial neural network is completed (step S1150). In this step, when it is determined that bit quantization for all layers of the artificial neural network is complete, the entire process is terminated. On the other hand, if there is a layer that has not yet been bit quantized among the layers of the artificial neural network, step S1110 is executed to perform bit quantization on the corresponding layer.

Here, in the method of selecting another layer from among the plurality of layers included in the artificial neural network in step S1110, (i) according to the arrangement order of the plurality of layers constituting the artificial neural network, the next layer of the previously selected layer is selected. A method of sequentially selecting (“forward bit quantization”, forward bit quantization), (ii) a method of selecting a previous layer of a previously selected layer in the reverse direction according to the arrangement order of a plurality of layers constituting an artificial neural network (“reverse bit quantization”) Quantization", backward bit quantization), (iii) A method of selecting a layer with the highest computational amount after the previously selected layer according to the order of computational amount among a plurality of layers constituting an artificial neural network ("high computational bit quantization", high computational cost bit quantization), or (iv) a method of selecting a layer with the smallest computational amount after the previously selected layer according to the order of computational amount among a plurality of layers constituting the artificial neural network (“low computational cost bit quantization”, low computational cost bit quantization) may be executed according to

In one embodiment, the accuracy of the artificial neural network is that after the artificial neural network learns a method of solving a given problem (eg, recognition of an object included in an image as input data), the method of solving the problem is presented in the reasoning step. It can mean probability. In addition, the target value used in the bit quantization method described above may represent the minimum accuracy to be maintained after bit quantization of the artificial neural network. For example, assuming that the target value is 90% accuracy, even after the parameter of the layer selected by bit quantization is reduced in bits, if the accuracy of the corresponding artificial neural network is 90% or more, additional bit quantization can be performed. For example, after performing the first bit quantization, if the accuracy of the artificial neural network is measured to be 94%, additional bit quantization can be performed. After the execution of the second bit quantization, if the accuracy of the artificial neural network is measured to be 88%, the result of the currently executed bit quantization is ignored, and the number of bits determined by the first bit quantization (that is, the number of bits to represent the data) may be determined as the final bit quantization result.

In an embodiment, according to a computational cost bit quantization method, in an artificial neural network including a plurality of layers, when a layer to be bit quantized is selected based on the amount of computation from among the plurality of layers, the amount of computation of each layer can be determined as follows. That is, when one addition operation performs n-bit and m-bit addition in a specific layer of the artificial neural network, the amount of operation of the corresponding operation is calculated as (n+m)/2. In addition, when a specific layer of the artificial neural network multiplies n bits by m bits, the amount of computation of the corresponding operation may be calculated as n x m. Accordingly, the computational amount of a specific layer of the artificial neural network may be a result of summing all the computational amounts of addition and multiplication executed by the layer.

In addition, according to the computational cost bit quantization method, a method of performing bit quantization by selecting a layer from among a plurality of layers based on the amount of computation in the artificial neural network is not limited to that shown in FIG. 11, but various Transformation is possible.

In another embodiment, in the embodiment shown in FIG. 11 , bit quantization of parameters for each layer may be performed separately for each of a weight and an activation map. For example, first, quantization is performed on the weight of the selected layer, and as a result, the weight has n bits. Separately, by performing bit quantization on the output activation data of the selected layer, the number of representation bits of the activation map data may be determined as m bits. Alternatively, quantization may be performed while allocating the same bits to the weight of the corresponding layer and the activation map data, and as a result, both the weight and the activation map data may be expressed with the same n bits.

12 is a flowchart illustrating a bit quantization method of an artificial neural network according to another embodiment of the present disclosure.

As shown, the bit quantization method 1200 of the artificial neural network may begin with the step of selecting a layer with the highest computational amount from among a plurality of layers included in the artificial neural network (S1210).

When the layer selection of the artificial neural network is completed in step S1210, the step of reducing the size of the data representation for the parameter of the selected layer in bits (S1220) may be performed. In an embodiment, when the size of the data of the selected layer is reduced in bits, the weight kernel quantization 1028 and the activation map quantization 1024 described with reference to FIGS. 4 to 10 may be performed.

After the execution of step S1220, a step S1230 of determining whether the accuracy of the artificial neural network reflecting the bit quantization results so far is greater than or equal to a predetermined target value may proceed. When it is determined in step S1230 that the accuracy of the artificial neural network is greater than or equal to the target value, the size of the data of the corresponding layer is set as the current bit quantization result, and the process proceeds to step S1210 and steps S1210 to S1230 are repeatedly executed. can That is, the process proceeds to step S1210, the computation amount is recalculated for all layers in the artificial neural network, and the layer with the highest computation amount is selected again based on this.

In step S1230, if the accuracy of the artificial neural network is not equal to or greater than the target value, the bit-reduced quantization of the currently selected layer is canceled, and the layer is excluded from the layer target selectable in the layer selection step S1210. Then, a layer with the next highest computational amount of the corresponding layer may be selected (step S1240). Next, the size of the data of the selected layer may be reduced in bits (step S1250).

In step S1260, it is determined whether the accuracy of the artificial neural network reflecting the bit quantization results so far is equal to or greater than a target value. If the accuracy of the artificial neural network is not higher than the target value, it is determined whether bit quantization for all layers of the artificial neural network is completed (S1270). If it is determined in step S1270 that bit quantization for all layers of the artificial neural network is complete, the entire bit quantization procedure is terminated. On the other hand, if it is determined in step S1270 that bit quantization for all layers of the artificial neural network is not completed, the process may proceed to step S1240.

If it is determined in step S1260 that the accuracy of the artificial neural network is greater than or equal to the target value, the process may proceed to step 1220 and subsequent procedures.

13 is a flowchart illustrating a bit quantization method of an artificial neural network having a plurality of layers according to another embodiment of the present disclosure.

As shown, the bit quantization method 1300 of an artificial neural network having a plurality of layers includes steps ( S1310 to S1350 ) of searching for accuracy variation points for each of all layers included in the artificial neural network. The method 1300 starts with a step (S1310) of initially fixing the bit size of data of all layers included in the artificial neural network to the maximum, and selecting one layer in which the accuracy change point is not searched.

When the selection of the layer of any artificial neural network is completed in step S1310, the step of reducing the size of data of the selected layer in bit units may proceed to step S1320. In an embodiment, when the size of the data of the selected layer is reduced in bits, the weight kernel quantization 1028 and the activation map quantization 1024 described with reference to FIGS. 4 to 10 may be performed.

After the execution of step S1320, a step S1330 of determining whether the accuracy of the artificial neural network reflecting the bit quantization results so far for the selected layer is equal to or greater than a predetermined target value may be performed. When it is determined in step S1330 that the accuracy of the artificial neural network is equal to or greater than the target value, the process proceeds to step S1320 to perform additional bit-reduced quantization on the currently selected layer.

In step S1330, if the accuracy of the artificial neural network is not equal to or greater than the target value, the number of data bits of the currently selected layer is set to the minimum number of bits that most recently satisfied the target value. Thereafter, it is determined whether the search for accuracy variation points for all layers of the artificial neural network is completed ( S1340 ). In this step, if the search for accuracy variation points for all layers is not completed, the process may proceed to step S1310. In step S1310, the bit size of data of all layers included in the artificial neural network is the maximum, and another layer in which the performance change point is not searched is selected.

If, in step S1340, it is determined that the search for the accuracy change point for all layers of the artificial neural network is completed, the bit quantization result corresponding to the accuracy change point for each layer of the artificial neural network may be reflected in the artificial neural network. (S1350). In one embodiment, in step S1350, the accuracy variation point of each layer of the artificial neural network determined according to the steps S1310 to S1340 described above (for example, deterioration of the accuracy of the artificial neural network in each layer occurs) point), set the layer to the bit size of the previous data.

In another embodiment, in step S1350, the layer is set to be larger than the size of the resource required for the calculation of the parameter immediately before the accuracy change point of each layer of the artificial neural network determined according to the steps S1310 to S1340 described above. do. For example, the number of bits of the parameter of each layer of the artificial neural network may be set to be 2 bits larger than the number of bits immediately before the accuracy change point. Then, the bit quantization method is executed on the artificial neural network having the size of data of each layer set in step 1350 ( S1360 ). The bit quantization method executed in step S1360 may include, for example, the method illustrated in FIG. 11 or FIG. 12 .

The bit quantization method of the artificial neural network according to the various embodiments described above is not limited to being executed on the weight kernel and feature map (or activation map) of each of a plurality of layers of the artificial neural network. In one embodiment, the bit quantization method of the present disclosure is first executed for the weight kernels (or weights) of all layers of the artificial neural network, and bit again for the feature maps of all layers of the artificial neural network in which the weight kernel quantization is reflected. Quantization may be performed. In another embodiment, bit quantization may be first performed on the feature maps of all layers of the artificial neural network, and bit quantization may be performed again on the kernels of all layers of the artificial neural network to which the feature map quantization is reflected.

In addition, the bit quantization method of the artificial neural network of the present disclosure is not limited to applying the same level of bit quantization to weight kernels of each layer of the artificial neural network. In an embodiment, the bit quantization method of the present disclosure may perform bit quantization in units of weight kernels of each layer of the artificial neural network, or individual bits so that each weight unit that is an element of each weight kernel may have a different bit. Quantization can also be performed.

Hereinafter, examples of execution results of the bit quantization method of an artificial neural network according to various embodiments of the present disclosure will be described with reference to the drawings.

14 is a graph illustrating an example of an amount of computation for each layer of an artificial neural network according to an embodiment of the present disclosure. The artificial neural network shown in FIG. 14 is an example of a convolutional neural network of the VGG-16 model including 16 layers, and each layer of the artificial neural network has a different amount of computation.

For example, since the second layer, the fourth layer, the sixth layer, the seventh layer, the ninth layer, and the tenth layer have the highest computational amount, a high computational cost bit quantization method may be used. In this case, bit quantization may be applied first. Also, after bit quantization is performed on the second, fourth, sixth, seventh, ninth, and tenth layers, bit quantization may be performed on the 14th layer having the next highest computational amount.

15 is a graph illustrating the number of bits per layer of an artificial neural network in which bit quantization is performed by a forward bit quantization method according to an embodiment of the present disclosure.

As described above, the forward quantization is a method of sequentially performing bit quantization from the frontmost layer (eg, from the layer from which input data is first received) based on the arrangement order of a plurality of layers included in the artificial neural network. to be. FIG. 15 shows the number of bits for each layer after forward quantization is applied to the artificial neural network of the VGG-16 model shown in FIG. 14 and the reduction rate of the computational amount of the artificial neural network by forward quantization. For example, when performing n-bit and m-bit addition, the amount of operation of the corresponding operation is calculated as (n+m)/2. In addition, when performing multiplication of n bits and m bits, the amount of operation of the corresponding operation can be calculated as n x m. Accordingly, the total amount of computation of the artificial neural network may be the result of summing all the computations of addition and multiplication executed in the corresponding artificial neural network.

As shown, when bit quantization is performed on the artificial neural network of the VGG-16 model using forward quantization, the number of bits in the layers arranged in front of the artificial neural network is relatively greatly reduced, and the number of bits arranged in the back of the artificial neural network is greatly reduced. The number of bits decreased slightly. For example, the number of bits in the first layer of the artificial neural network is reduced to 12 bits, the number of bits in the second layer and the third layer is reduced to 9 bits, respectively, while the number of bits in the 16th layer is reduced to 13 bits, , the number of bits in the 15th layer is reduced to only 15 bits. As such, when forward quantization is sequentially applied from the first layer to the 16th layer of the artificial neural network, the reduction rate of the total amount of computation of the artificial neural network is calculated to be 56%.

16 is a graph illustrating the number of bits per layer of an artificial neural network in which bit quantization is performed by a backward bit quantization method according to an embodiment of the present disclosure.

The reverse quantization is a method of sequentially performing bit quantization from the rearmost layer (eg, from the layer from which output data is finally output) based on the arrangement order of a plurality of layers included in the artificial neural network. FIG. 16 shows the number of bits for each layer after applying reverse quantization to the artificial neural network of the VGG-16 model shown in FIG. 14 and the reduction rate of the amount of computation of the artificial neural network by reverse quantization.

As shown, when bit quantization is performed on the artificial neural network of the VGG-16 model using reverse quantization, the number of bits in the layers arranged behind the artificial neural network is reduced relatively significantly, whereas the number of bits in the layers arranged in front of the artificial neural network is reduced. The number of bits decreased slightly. For example, the number of bits of the first layer, the second layer, and the third layer is reduced to 15 bits, respectively, and the number of bits of the fourth layer is reduced to 14 bits, while the number of bits of the 16th layer is reduced to 9 bits and the number of bits in the 15th layer is reduced to 15 bits. As such, when reverse quantization is sequentially applied from the first layer to the 16th layer of the artificial neural network, the reduction rate of the total amount of computation of the artificial neural network was calculated to be 43.05%.

17 is a graph illustrating the number of bits per layer of an artificial neural network in which bit quantization is performed by a high computational cost layer first bit quantization method according to an embodiment of the present disclosure.

High-computational layer-priority quantization (or high-computational quantization) is a method of sequentially performing bit quantization from a layer with a high computational amount among a plurality of layers included in the artificial neural network. FIG. 17 shows the number of bits for each layer after high computational quantization is applied to the artificial neural network of the VGG-16 model shown in FIG. 14 and the reduction rate of the computational amount of the artificial neural network by high computational quantization.

As shown, when bit quantization is performed on the artificial neural network of the VGG-16 model using high computational quantization, the number of bits in layers with high computational amount among a plurality of layers of the artificial neural network is reduced relatively significantly. For example, the number of bits in the second layer and the tenth layer is reduced to 5 bits and 6 bits, respectively, while the number of bits in the first layer is reduced to 14 bits. As such, when high computational quantization is applied to the layers of the artificial neural network in the order of computational amount, the reduction rate of the total computational amount of the artificial neural network was calculated to be 70.70%.

18 is a graph illustrating the number of bits per layer of an artificial neural network in which bit quantization is performed by a low computational cost bit quantization method according to an embodiment of the present disclosure.

Low-computational layer-priority quantization (or low-computational quantization) is a method of sequentially performing bit quantization from a layer with a low computational amount among a plurality of layers included in the artificial neural network. FIG. 18 shows the number of bits for each layer after applying low computational quantization to the artificial neural network of the VGG-16 model shown in FIG. 14 and the reduction rate of the computational amount of the artificial neural network by low computational quantization.

As shown, even when bit quantization is performed on the artificial neural network of the VGG-16 model using low computational quantization, the number of bits in layers with high computational amount among a plurality of layers of the artificial neural network is reduced relatively significantly. For example, the number of bits in the sixth layer and the seventh layer is reduced to 6 bits and 5 bits, respectively, while the number of bits in the first layer is reduced to 13 bits. As described above, when low computational quantization was applied to the layers of the artificial neural network in the order of computational amount, the reduction rate of the computational amount of the entire artificial neural network was calculated to be 49.11%.

Hereinafter, hardware implementation examples of the artificial neural network to which bit quantization is applied according to various embodiments of the present disclosure described above will be described in detail. When a convolutional neural network including a plurality of layers is implemented as hardware, the weight kernel may be arranged outside and/or inside the processing unit for executing convolution of the convolutional layers.

In one embodiment, the weight kernel may be stored in a memory (eg, register, butter, cache, etc.) separate from the processing unit for performing the convolution of the convolutional layer. In this case, after reducing the number of bits of element values of the weight kernel by applying bit quantization to the weight kernel, the size of the memory may be determined according to the number of bits of the weight kernel. In addition, the bit width of a multiplier or adder arranged in a processing unit that receives element values of the weight kernel stored in the memory and element values of the input feature map and executes multiplication and/or addition operations. Also, it can be designed according to the number of bits according to the result of bit quantization.

In another embodiment, the weight kernel may be implemented in a hard-wired form in a processing unit for performing convolution of a convolutional layer. In this case, after reducing the number of bits of element values of the weight kernel by applying bit quantization to the weight kernel, a hard wire representing each of the element values of the weight kernel according to the number of bits of the weight kernel may be implemented in the processing unit. In addition, the bit size of a multiplier or adder arranged in a processing unit that receives the element values of the hardwired weight kernel and element values of the input feature map and executes multiplication and/or addition operations is also dependent on the number of bits according to the result of bit quantization. can be designed accordingly.

19 to 21 described below are diagrams illustrating a hardware implementation example of an artificial neural network including a plurality of layers according to another embodiment of the present disclosure. The bit quantization method and system of an artificial neural network including a plurality of layers according to the present disclosure, by applying the present disclosure to any ANN (Artificial Neural Network) calculation system such as CPU, GPU, FPGA, ASIC, required amount of computation, bit size of operator, memory can be reduced. In addition, although the embodiment is shown based on an integer in this example, it may also be implemented as a floating point operation.

19 is a diagram illustrating a hardware implementation example of an artificial neural network according to an embodiment of the present disclosure. The illustrated artificial neural network represents an example in which the convolution processing unit 1900 of the convolutional layer of the convolutional artificial neural network is implemented in hardware. Here, the convolution layer will be described on the assumption that a 3x3x3 weight kernel is applied to a part (3x3x3 data) on the input feature map to perform convolution. The size and number of weight kernels of each layer may be different depending on the application field and the number of input/output feature map channels.

As shown, the weight kernel may be stored in a weight kernel cache 1910 separate from the processing unit 1930 for executing convolution of the convolutional layer. In this case, after the bit quantization is applied to the weight kernel _{to reduce the number of bits of the element values (w 1} , w ₂ , ..., w ₉ ) of the weight kernel, the size of the cache is adjusted according to the number of bits of the weight kernel. can decide In addition, the bit size of a multiplier or adder arranged in the processing unit 1930 that receives element values of the weight kernel stored in the memory and element values of the input feature map and executes multiplication and/or addition operations is also determined according to the result of bit quantization. It can be designed according to the number of bits of the weighted kernel element value.

According to an embodiment, the input feature map cache 1920 may receive and store a portion of input data (a portion corresponding to the size of the weight kernel) as input. The weight kernel traverses the input data phase, and the input feature map cache 1920 may sequentially receive and store a portion of the input data corresponding to the location of the weight kernel. _{Part of the input data (x 1} , x ₂ , ..., x ₉ ) stored in the input feature map cache 1920 and some element values of the weight kernel stored in the weight kernel cache 1910 (w ₁ , w ₂ , . .., w ₉ ) is input to the corresponding multiplier 1932, respectively, and multiple multiplication is executed. The result values of multiple multiplication by the multiplier 1932 are summed by the tree adder 1934 and input to the adder 1940 . When the input data is composed of multiple channels (eg, when the input data is an RGB color image), the adder 1940 is the sum of the value stored in the accumulator 1942 (the initial value is 0) and the input specific channel. can be added and stored in the accumulator 1942 again. The sum value stored in the accumulator 1942 may be input to the accumulator 1942 by adding it back to the sum value of the adder 1940 for the next channel. The summing process of the adder 1940 and the accumulator 1942 may be performed for all channels of input data, and the total sum may be input to the output activation map cache 1550 . The convolution procedure described above may be repeated for a weight kernel and a portion of input data corresponding to a traversal position on input data of the weight kernel.

As described above, when the element values of the weight kernel are stored in the weight kernel cache 1910 arranged outside the processing unit 1930, the number of bits of the weight kernel element values is reduced by bit quantization according to the present disclosure. Accordingly, there is an effect that the size of the weight kernel cache 1910 and the size of the multiplier and the adder of the processing unit 1930 can be reduced. Also, as the size of the processing unit 1930 decreases, the operation speed and power consumption of the processing unit 1930 may also decrease.

20 is a diagram illustrating an example of hardware implementation of an artificial neural network according to another embodiment of the present disclosure.

The illustrated artificial neural network shows an example in which the convolution processing apparatus 2000 of the convolutional layer of the convolutional artificial neural network is implemented in hardware. Here, the convolution layer performs convolution by applying a weight kernel of 3x3x3 size to a portion (data of size 3x3x3) on the input activation map.

As shown, the weight kernel may be stored in the weight kernel cache 2010 separated from the processing unit 2030 for executing convolution of the convolutional layer. In this case, after the bit quantization is applied to the weight kernel _{to reduce the number of bits of the element values (w 1} , w ₂ , ..., w ₉ ) of the weight kernel, the size of the cache is adjusted according to the number of bits of the weight kernel. can decide In addition, the bit size of a multiplier or adder arranged in the processing unit 2030 that receives element values of the weight kernel stored in the memory and element values of the input activation map (or feature map) and executes multiplication and/or addition operations in bits It can be designed according to the number of bits of the weight kernel element value according to the result of quantization.

According to an embodiment, the input activation map cache 2020 may receive and store a portion (a portion corresponding to the size of the weight kernel) on input data composed of multiple channels (eg, three RGB channels). The weight kernel traverses the input data phase, and the input activation map cache 2020 may sequentially receive and store a portion of the input data corresponding to the location of the weight kernel. _{A portion of input data (x 1} , x ₂ , ..., x ₂₇ ) stored in the input activation map cache 2020 and element values of the weight kernel stored in the weight kernel cache 2010 (w ₁ , w ₂ , . .., w ₂₇ ) is input to the corresponding multiplier, and multiple multiplication is executed. At this time, the kernel element values (w ₁ , w ₂ , ..., w _{9 ) of the weight kernel cache 2010 and the portion (x 1} ) of the first channel of the input data stored in the input activation map weight cache 2020 , x ₂ , ..., x ₉ ) are input to the first convolution processing unit 2032 . In addition, the weight kernel element values (w ₁₀ , w ₁₁ , ..., w ₁₈ ) of the weight kernel cache 2010 and the second channel portion of the input data stored in the input activation map cache 2020 (x ₁₀ , x ₁₁ , ..., x ₁₈ ) are input to the second convolution processing unit 2034 . In addition, the weight kernel element values (w ₁₉ , w ₂₀ , ..., w ₂₇ ) of the weight kernel cache 2010 and the third channel part of the input data stored in the input activation map cache 2020 (x ₁₉ , x ₂₀ , ..., x ₂₇ ) are input to the third convolution processing unit 2036 .

Each of the first convolution processing unit 2032 , the second convolution processing unit 2034 , and the third convolution processing unit 2036 may operate in the same manner as the processing unit 1930 illustrated in FIG. 19 . The result values of the convolutions calculated by each of the first convolution processing unit 2032 , the second convolution processing unit 2034 , and the third convolution processing unit 2036 are summed by the tree adder 2038 and output It may be input to the activation map cache 2040 .

As described above, when the element values of the weight kernel are stored in the weight kernel cache 2010 arranged outside the processing unit 2030, the number of bits of the weight kernel element values is reduced by bit quantization according to the present disclosure. Accordingly, there is an effect that the size of the weight kernel cache 2010 and the size of the multiplier and adder of the processing unit 2030 can be reduced. Also, as the size of the processing unit 2030 decreases, the operation speed and power consumption of the processing unit 2030 may also decrease.

21 is a diagram illustrating an example of hardware implementation of an artificial neural network according to another embodiment of the present disclosure.

The illustrated artificial neural network shows an example in which the convolution processing device 2200 of the convolutional layer of the convolutional artificial neural network is implemented in hardware. Here, the convolution layer performs convolution by applying a weight kernel of 3x3x3 size to a portion (data of size 3x3x3) on the input activation map.

As shown, the weight kernel may be implemented in hardwired form within the processing unit 2220 for performing convolution of the convolutional layer. In this case, after the bit quantization is applied to the weight kernel _{to reduce the number of bits of the element values (w 1_K} , w _{2_K} , ..., w _{27_K} ) of the weight kernel, the size of the cache depends on the number of bits of the weight kernel. can be decided In addition, arranged in the processing unit 2030 that receives element values of the weight kernel implemented as wires in the processing unit 2220 and element values of the input activation map (or feature map) and executes multiplication and/or addition operations. The bit size of the multiplier or adder may also be designed according to the number of bits of the weighted kernel element value according to the result of bit quantization.

According to an embodiment, the input activation map cache 2210 may receive and store a portion (a portion corresponding to the size of the weight kernel) on input data composed of multiple channels (eg, three RGB channels). The weight kernel traverses the input data phase, and the input activation map cache 2210 may sequentially receive and store a portion of the input data corresponding to the location of the weight kernel. _{A portion of input data (x 1} , x ₂ , ..., x ₂₇ ) stored in the input activation map cache 2210 _{and element values (w 1_K} , w _{2_K} ) of the weight kernel implemented as wires in the processing unit 2220 . , ... w _{27_K} ) are input to the corresponding multiplier, respectively, and multiple multiplication is executed. At this time, the weight kernel element values w _{1_K} , w _{2_K} , ..., w _{9_K} implemented as wires in the processing unit 2220 and the input data stored in the input activation map cache 2210 are part of the first channel (x ₁ , x ₂ , ..., x ₉ ) is input to the first convolution processing unit 2222 . In addition, the weight kernel element values (w _{10_K} , w _{11_K} , ..., w _{18_K} ) implemented as wires in the processing unit 2220 and the second channel portion of the input data stored in the input activation map cache 2210 ( x ₁₀ , x ₁₁ , ..., x ₁₈ ) are input to the second convolution processing unit 2224 . In addition, the weight kernel element values (w _{19_K} , w _{20_K} , ..., w _{27_K} ) of the weight kernel cache 2210 and the third channel part of the input data stored in the input activation map cache 2210 (x ₁₉ , x ₂₀ , ..., x ₂₇ ) are input to the third convolution processing unit 2226 .

The result values of the convolutions calculated by each of the first convolution processing unit 2222 , the second convolution processing unit 2224 , and the third convolution processing unit 2226 are summed by the tree adder 2228 and output It may be input to the activation map cache 2230 .

As described above, when the element values of the weight kernel are implemented in a hard-wired form in the processing unit 2220, the number of bits of the weight kernel element values can be reduced by bit quantization according to the present disclosure, and accordingly, processing The number of wires implemented inside the unit 2220 and the size of the multipliers and adders of the processing unit 2220 can be reduced. Also, as the size of the processing unit 2220 decreases, the operation speed and power consumption of the processing unit 2220 may also decrease.

22 is a diagram illustrating a configuration of a system for performing bit quantization on an artificial neural network according to an embodiment of the present disclosure.

As shown, the system 2300 may include a parameter selection module 2310 , a bit quantization module 2320 , and an accuracy determination module 2330 . The parameter selection module 2310 may analyze the input configuration information of the artificial neural network. The configuration information of the artificial neural network includes the number of layers included in the artificial neural network, the function and role of each layer, information about input and output data of each layer, the type and number of multiplications and additions performed by each layer, and by each layer. The type of activation function to be executed, the type and composition of the weight kernel to which each layer is input, the size and number of weight kernels belonging to each layer, the size of the output feature map, the initial value of the weight kernel (e.g., weights set by real numbers) kernel element values), but is not limited thereto. The configuration information of the artificial neural network may include information of various components according to the type of the artificial neural network (eg, a convolutional artificial neural network, a recurrent artificial neural network, a multilayer perceptron, etc.).

The parameter selection module 2310 may select at least one parameter or parameter group to be quantized in the corresponding artificial neural network with reference to the input artificial neural network configuration information. How one parameter (or data) or parameter group is selected in the artificial neural network may be determined according to the effect of the selected parameter on the overall performance or computational amount (or the amount of resources required for hardware implementation) of the artificial neural network. The parameter selection may be performed by selection of one weight, one feature map and activation map, one weight kernel, all weights belonging to one layer, all feature maps belonging to one layer, or activation map.

In one embodiment, in the case of the convolutional artificial neural network (CNN) 400 described with reference to FIGS. 4 to 10 described above, the convolutional layer 420 and/or the fully connected layer 440 is the CNN 400 Since it has a large effect on the overall performance or amount of computation, a weight kernel or a feature map/activation map of at least one of these layers 420 and 440 may be selected as one parameter to be quantized.

In an embodiment, by selecting at least one of a plurality of layers included in the artificial neural network, all weight kernels in the layer or all activation map data of the layer may be set as one parameter group. The selection method includes: may be determined according to an effect on the overall performance or amount of computation of the artificial neural network, but is not limited thereto, and may include one of various methods. For example, the selection of at least one layer among a plurality of layers included in the artificial neural network is sequentially performed from a first layer from which input data is received to a subsequent layer according to the arrangement order of the plurality of layers constituting the artificial neural network. (ii) a method of sequentially selecting from the last layer in which the final output data is generated to the previous layer according to the arrangement order of the plurality of layers constituting the artificial neural network, (iii) the plurality of layers constituting the artificial neural network. It may be performed according to a method of selecting a layer with the highest amount of computation from among the layers, or (iv) a method of selecting a layer with the least amount of computation from among a plurality of layers constituting the artificial neural network.

When the selection of a data target to be quantized by the artificial neural network is completed by the parameter selection module 2310 , information on the selected data is input to the bit quantization module 2320 . The bit quantization module 2320 may reduce the data representation size of the corresponding parameter in units of bits with reference to the inputted information of the selected parameter. The resource required for the operation of the selected parameter may include, but is not limited to, a memory for storing the selected parameter or a data path for transmitting the selected parameter.

In an embodiment, when the bit quantization module 2320 reduces the data size of the selected parameter in bits, the weight kernel quantization and/or activation map quantization described with reference to FIGS. 4 to 13 may be performed.

When the bit quantization module 2320 completes bit quantization for the selected parameter, the bit quantized artificial neural network information is transmitted to the accuracy determination module 2330 . The accuracy determination module 2330 may reflect bit-quantized information of the artificial neural network to the configuration information of the artificial neural network input to the system 2300 . The bit quantization module 2320 may determine whether the accuracy of the artificial neural network is equal to or greater than a predetermined target value, based on the configuration information of the artificial neural network in which the bit-quantized information of the artificial neural network is reflected. For example, the accuracy determination module 2330 decreases the size of the data representing the data of the selected parameter in the artificial neural network in bits, and then the accuracy of the output result of the corresponding artificial neural network (eg, the inference result of the artificial neural network) is If it is more than a predetermined target value, it can be predicted that the overall performance of the artificial neural network can be maintained even if additional bit quantization is performed.

Accordingly, when the accuracy determination module 2330 determines that the accuracy of the artificial neural network is equal to or greater than the target value, it transmits a control signal to the parameter selection module 2310 so that the parameter selection module 2310 selects another one included in the artificial neural network. Allows you to select a parameter or parameter group. Here, the method of selecting one parameter in the artificial neural network is (i) a method of sequentially selecting the next parameter of the previously selected parameter according to the arrangement order of each parameter or parameter group constituting the artificial neural network (“forward bit quantization”) ", forward bit quantization), (ii) a method of selecting a previous parameter of a previously selected parameter in the reverse direction according to the arrangement order of parameters or parameter groups constituting an artificial neural network ("reverse bit quantization", backward bit quantization), ( iii) A method of selecting a parameter with the highest computational amount after the previously selected parameter according to the order of computational amount among a plurality of parameters constituting the artificial neural network (“high computational cost bit quantization”), or (iv) artificial According to the order of the computational amount among a plurality of parameters constituting the neural network, it may be executed according to a method of selecting a parameter with the smallest computational amount next to the previously selected parameter (“low computational cost bit quantization”).

On the other hand, if the accuracy determination module 2330 determines that the accuracy of the artificial neural network is not equal to or greater than the target value, it may determine that the accuracy of the artificial neural network is reduced due to bit quantization performed on the currently selected parameter. Accordingly, in this case, the number of bits determined by the bit quantization performed immediately before may be determined as the final number of bits. In one embodiment, the accuracy of the artificial neural network is the probability of presenting the correct answer to the problem in the reasoning step after the artificial neural network learns a method for solving a given problem (eg, recognition of an object included in an image as input data) can mean In addition, the target value used in the bit quantization method described above can be expressed with the minimum accuracy to be maintained after bit quantization of the artificial neural network. For example, assuming that the threshold is 90%, even after reducing the size of the memory for storing the parameters of the layer selected by bit quantization in bits, if the accuracy of the corresponding artificial neural network is 90% or more, additional bit quantization is performed. can run For example, after performing the first bit quantization, if the accuracy of the artificial neural network is measured to be 94%, additional bit quantization can be performed. After the execution of the second bit quantization, if the accuracy of the artificial neural network is measured to be 88%, it is possible to ignore the result of the currently executed bit quantization and determine the number of data representation bits determined by the first bit quantization as the final bit quantization result. have.

In an embodiment, according to a computational cost bit quantization method, when a parameter or parameter group to be bit quantized is selected based on the computational amount, the computational amount of each parameter may be determined as follows. That is, when the sum of n bits and m bits is performed in a specific operation of the artificial neural network, the amount of operation of the corresponding operation is calculated as (n+m)/2. In addition, when performing multiplication of n bits and m bits in a specific operation of the artificial neural network, the amount of operation of the corresponding operation can be calculated as n x m. Accordingly, the amount of computation for a specific parameter of the artificial neural network may be a result of summing all computations of addition and multiplication performed on the parameter.

A method of selecting a specific parameter or parameter group in such bit quantization includes weight data or feature map and activation map data belonging to each layer, or each weight kernel belonging to one layer, or each weight data in one weight kernel. can be selected as individual parameter groups.

For reference, the components illustrated in FIG. 22 according to an embodiment of the present disclosure may be implemented as software or hardware components such as Field Programmable Gate Array (FPGA) or Application Specific Integrated Circuit (ASIC).

However, the term 'components' is not limited to software or hardware, and each component may be configured to reside in an addressable storage medium or to reproduce one or more processors.

Thus, as an example, a component includes components such as software components, object-oriented software components, class components, and task components, and processes, functions, properties, procedures, sub It includes routines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables.

Components and functions provided within the components may be combined into a smaller number of components or further divided into additional components.

Embodiments of the present disclosure may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer-readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism, and includes any information delivery media.

Although the present disclosure has been described with reference to some embodiments herein, it should be understood that various modifications and changes may be made without departing from the scope of the present disclosure as understood by those skilled in the art to which the present invention pertains. something to do. Further, such modifications and variations are intended to fall within the scope of the claims appended hereto.

100: artificial neural network 110_1 to 110_N: layer
200: artificial neural network 210: input data
220: input layer 230: hidden layer
240: output layer 400: artificial neural network
410: input data 420: convolutional layer (CONV)
430: subsampling layer (SUBS) 440: fully connected layer (FC)
450: output data 510: input data
520: weight kernel 530, 260: row
540, 270: column 550, 280: depth (channel)
610: first weight kernel 620: first result value
630: first activation map 710: second weight kernel
720: second result value 730: second activation map
810: input data 812: multiple product
814: weight kernel 816: sum
818: output data 820: first traversal section
822: second traversal section 824: first feature map value
826: second feature map value 910: input data
912: multiple product 914: weight kernel
916: sum 918: output data
1010: input data 1012: multiple product
1014: weight kernel 1016: quantization
1018: quantization weight kernel 1020: sum
1022: output data 1024: quantization
1026: quantization activation map 1028: weighted kernel quantization
1030: Activation map quantization

Claims (19)

In the bit quantization method of a multi-layered artificial neural network executed by a system,
calculating an amount of computation for each of a plurality of layers of the multi-layered artificial neural network;
selecting at least one layer from among the plurality of layers in the order of the layer having the highest computational amount;
a bit quantization step of reducing the size of the data representation of the selected at least one layer parameter in units of bits;
determining whether the accuracy of the multi-layered artificial neural network is greater than or equal to a target value after the bit quantization step; and
and performing the bit quantization step again when the accuracy of the multi-layered artificial neural network is equal to or greater than the target value. In the bit quantization method of a multi-layered artificial neural network executed by a system,
calculating an amount of computation for each of a plurality of layers of the multi-layered artificial neural network;
selecting at least one layer from among the plurality of layers in the order of the layers having the lowest computational amount;
a bit quantization step of reducing the size of the data representation of the selected at least one layer parameter in units of bits;
determining whether the accuracy of the multi-layered artificial neural network is greater than or equal to a target value after the bit quantization step; and
and performing the bit quantization step again when the accuracy of the multi-layered artificial neural network is equal to or greater than the target value. In the bit quantization method of a multi-layered artificial neural network executed by a system,
The bit size of the data for the parameters of the plurality of layers of the multi-layered neural network is fixed to the maximum, and the accuracy change point, which is the point at which the deterioration of the accuracy of the artificial neural network occurs in each layer, is not searched at least selecting one layer;
a bit quantization step of reducing the size of the data representation of the selected at least one layer parameter in units of bits;
determining whether the accuracy of the multi-layered artificial neural network is greater than or equal to a target value after the bit quantization step; and
and performing the bit quantization step again when the accuracy of the multi-layered artificial neural network is equal to or greater than the target value. According to any one of claims 1 to 3,
When the accuracy of the multi-layered artificial neural network is less than the target value, determining the size of the data representation for the parameter of the selected layer that satisfies the accuracy above the target value as the final number of bits for the parameter of the selected layer. Including method. 5. The method of claim 4,
Selecting at least one layer in which the final number of bits for a parameter is not determined among the plurality of layers, and repeatedly executing the bit quantization step to determine the final number of bits of the at least one layer in which the selected final number of bits is not determined A method further comprising a step. 4. The method according to any one of claims 1 to 3,
The method according to claim 1, wherein the bit quantizing step of decreasing in bits is configured to decrease in units of 1 bit. 4. The method according to any one of claims 1 to 3,
The method, characterized in that the parameter of the selected at least one layer comprises at least one of weight data, feature map data, and activation map data. 4. The method according to any one of claims 1 to 3,
wherein the bit quantizing step performs bit quantization to reduce a storage size of at least one of a buffer memory, a register memory, and a cache memory configured to store the parameter of the selected at least one layer. 4. The method according to any one of claims 1 to 3,
The number of bits of the multiplier and the adder of the processing unit for processing the bit-quantized multi-layered artificial neural network is designed to correspond to the number of bits according to the result of the bit quantization step, the method. In the bit quantization method of a multi-layered artificial neural network executed by a system including a processor,
calculating, through the processor, an amount of computation for each layer of the multi-layered artificial neural network;
selecting, through the processor, the weight data of the layer in an order of weight data of a layer having a high computational amount or weighting data of a layer having a low computational amount;
a bit quantization step of reducing, by the processor, a size of a data representation of the selected weight data in bits;
determining, through the processor, whether the accuracy of the multi-layered artificial neural network is equal to or greater than a target value after the bit quantization step; and
When, through the processor, the accuracy of the multi-layered artificial neural network is equal to or greater than the target value, performing the bit quantization step again. In the bit quantization method of a multi-layered artificial neural network executed by a system including a processor,
calculating, through the processor, an amount of computation for each layer of the multi-layered artificial neural network;
selecting, through the processor, the feature map data of the layer in the order of the feature map data of the layer with the high computational amount or the feature map data of the layer with the low computational amount;
a bit quantization step of reducing, by the processor, a size of a data representation of the selected feature map data in units of bits;
determining, through the processor, whether the accuracy of the multi-layered artificial neural network is equal to or greater than a target value after the bit quantization step; and
When, through the processor, the accuracy of the multi-layered artificial neural network is equal to or greater than the target value, performing the bit quantization step again.
In the bit quantization method of a multi-layered artificial neural network executed by a system including a processor,
calculating, through the processor, an amount of computation for each layer of the multi-layered artificial neural network;
selecting, through the processor, the activation map data of the layer in an order of activation map data of a layer having a high computational amount or in an order of activation map data of a layer having a low computational amount;
a bit quantization step of reducing, by the processor, a size of a data representation of the selected activation map data in units of bits;
determining, through the processor, whether the accuracy of the multi-layered artificial neural network is equal to or greater than a target value after the bit quantization step; and
When, through the processor, the accuracy of the multi-layered artificial neural network is equal to or greater than the target value, performing the bit quantization step again. In the bit quantization method of a multi-layered artificial neural network executed by a system including a processor,
Through the processor, the bit size of data for parameters including one of weight data, feature map data, and activation map data of each layer of the multi-layered artificial neural network is fixed to the maximum, and in each layer, the artificial neural network selecting one parameter for which the search for an accuracy variation point, which is a point at which deterioration of accuracy of , is not performed; a bit quantization step of reducing the size of the data representation for the selected one parameter in bits;
determining, through the processor, whether the accuracy of the multi-layered artificial neural network is equal to or greater than a target value after the bit quantization step; and
When, through the processor, the accuracy of the multi-layered artificial neural network is equal to or greater than the target value, performing the bit quantization step again.
14. The method according to any one of claims 10 to 13,
When the accuracy of the multi-layered artificial neural network is less than the target value, determining the size of the data representation for the selected at least one parameter that satisfies the accuracy greater than or equal to the target value as the final number of bits for the selected at least one parameter; A method further comprising:
15. The method of claim 14,
selecting at least one parameter for which the final number of bits for the plurality of parameters is not determined, and repeatedly executing the bit quantization step to determine the final number of bits of the at least one parameter for which the selected final number of bits is not determined; further comprising the method.
14. The method according to any one of claims 10 to 13,
The method according to claim 1, wherein the bit quantizing step of decreasing in bits is configured to decrease in units of 1 bit.
14. The method according to any one of claims 10 to 13,
wherein the bit quantizing step is bit quantized to reduce a storage size of at least one of a buffer memory, a register memory, and a cache memory configured to store the selected at least one parameter.
14. The method according to any one of claims 10 to 13,
The number of bits of the multiplier and the adder of the processing unit for processing the bit-quantized multi-layered artificial neural network is designed to correspond to the number of bits according to the result of the bit quantization step, the method.
14. The method of claim 13,
The method further comprising the step of setting the number of bits of the parameter of each layer of the multi-layered artificial neural network to be 2 bits larger than the number of bits immediately before the accuracy change point.

Images