KR20240093407A - Method for bit quantization of artificial neural network and convultion processor - Google Patents

Abstract

The present disclosure provides a method for bit quantization of an artificial neural network. This method includes the steps of (a) selecting one parameter or one group of parameters to be quantized in an artificial neural network; (b) a bit quantization step of reducing the data representation size for the selected parameter or parameter group in bits; (c) determining whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value; (d) If the accuracy of the artificial neural network is greater than or equal to the target value, repeating steps (a) to (c) may be included.

Description

Bit quantization method and convolution processing device of artificial neural network {METHOD FOR BIT QUANTIZATION OF ARTIFICIAL NEURAL NETWORK AND CONVULTION PROCESSOR}

The present disclosure relates to the hardware of an artificial neural network and a convolution processing device, and more specifically, to the hardware of an artificial neural network and a convolution processing device that can reduce performance and memory usage while maintaining the substantial accuracy of the artificial neural network. It's about.

An artificial neural network is a computer structure that models the biological brain. In an artificial neural network, nodes corresponding to neurons in the brain are interconnected, and the strength of synaptic connections between neurons is expressed as a weight. Artificial neural networks construct a model with the ability to solve a given problem by changing the strength of synaptic connections between artificial neurons (nodes) through learning.

Artificial neural network may, in a narrow sense, refer to a multi-layered perceptron, a type of feedforward neural network, but is not limited thereto, and may include a radial basis function network, a self-organizing neural network, etc. It may include various types of neural networks, such as self-organizing network and recurrent neural network.

Recently, a deep neural network with a multi-layer structure has been widely used as a technology for image recognition, and a representative example of a deep neural network with a multi-layer structure is a convolutional neural network (CNN). In the case of a general multi-layer forward transfer neural network, the input data is limited to one-dimensional form. When image data consisting of two or three dimensions is flattened into one-dimensional data, spatial information is lost, and the spatial information of the image is lost. It may be difficult to train a neural network in this state. However, convolutional neural networks are capable of learning visual information while maintaining two-dimensional or three-dimensional spatial information.

Specifically, the convolutional neural network effectively recognizes features of adjacent images while maintaining the spatial information of the image, and includes a Max Pooling process that collects and strengthens the features of the extracted image, thereby recognizing patterns in visual data. It is effective for However, deep neural networks with a multi-layer structure, such as these convolutional neural networks, use a deep layer structure to provide high recognition performance, but the structure is very complex and requires a large amount of computation and a large amount of memory. In a deep neural network with a multi-layer structure, most operations that occur internally are executed using multiplication and addition (or accumulation), as the number of connections between nodes in the artificial neural network is large and parameters that require multiplication (e.g., weight data, features) Since the number of data (map data, activation map data, etc.) is large, a large amount of computation is required during the learning process or recognition process.

As discussed above, a large amount of computation and memory are required during the learning and recognition process of a deep neural network with a multi-layer structure, such as a convolutional neural network. As a method of reducing the amount of computation and memory of a deep neural network with a multi-layer structure, a bit quantization method can be used, which reduces the data representation size of the parameters used in the computation of the artificial neural network in bits. The existing bit quantization method uses uniform bit quantization, which quantizes all parameters of the artificial neural network to the same number of bits. However, the existing uniform bit quantization method uses the number of bits for each parameter used in the artificial neural network. There is a problem with not accurately reflecting the impact of changes on overall performance.

Embodiments disclosed in this specification quantize each parameter data constituting the artificial neural network or parameter data grouped according to a specific standard to a specific number of bits in order to maintain artificial intelligence accuracy while improving overall performance in the artificial neural network. We would like to provide a method and system for doing so.

According to an embodiment of the present disclosure, a method for bit quantization of an artificial neural network is provided. This method includes: (a) selecting at least one parameter among a plurality of parameters used in an artificial neural network; (b) a bit quantization step of reducing the size of data required for the operation for the selected parameter in bit units; (c) determining whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value; (d) If the accuracy of the artificial neural network is greater than or equal to the target value, further reducing the number of bits in the data representation of the parameter by repeatedly executing steps (b) to (c) for the parameter. there is. In addition, this method includes (e) when the accuracy of the artificial neural network is less than the target value, restoring the number of bits of the parameter to the number of bits when the accuracy of the artificial neural network was greater than the target value, and then (a) It further includes repeating steps to (d).

According to an embodiment of the present disclosure, a method for bit quantization of an artificial neural network is provided. The method includes the steps of: (a) selecting, by a parameter selection module, at least one layer among the plurality of layers; (b) a bit quantization step of reducing, by a bit quantization module, the size of the data representation for the parameters of the selected layer in bits; (c) determining, by an accuracy determination module, whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value; and (d) repeating steps (a) to (c) when the accuracy of the artificial neural network is greater than or equal to the target value.

According to an embodiment of the present disclosure, a method for bit quantization of an artificial neural network is provided. This method includes the steps of: (a) selecting one or more data or one or more groups of data among weight, feature map, and activation map data in the artificial neural network by a parameter selection module; (b) a bit quantization step of reducing the data representation size for the selected data in bits by a bit quantization module; (c) measuring whether the artificial intelligence accuracy of the artificial neural network is greater than or equal to a target value; and (d) repeatedly executing steps (a) to (c) until there is no more data to be quantized among the data of the artificial neural network.

According to an embodiment of the present disclosure, a method for bit quantization of an artificial neural network is provided. The method includes training the artificial neural network according to one or more parameters of the artificial neural network; performing bit quantization on one or more parameters of the artificial neural network according to the bit quantization method of the artificial neural network according to the above embodiments; and training the artificial neural network according to one or more parameters of the artificial neural network on which the bit quantization has been performed.

According to another embodiment of the present disclosure, a bit quantization system for an artificial neural network is provided. The system includes a parameter selection module for selecting at least one parameter within the artificial neural network; a bit quantization step of reducing the size of the data representation of the selected parameter in bits; It may include an accuracy determination module that determines whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value. When the accuracy of the artificial neural network is greater than the target value, the accuracy determination module controls the parameter selection module and the bit quantization module to maintain the accuracy of the artificial neural network above the target value, and each of the plurality of parameters Quantization can be performed to have the minimum number of bits.

According to one embodiment of the present disclosure, a bit quantization system for an artificial neural network is provided. This system includes a parameter selection module that selects at least one layer among a plurality of layers constituting the artificial neural network; a bit quantization module that reduces the size of the data representation for the parameter of the selected layer in bits; and an accuracy determination module that determines whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value, and the accuracy determination module determines whether the accuracy of the artificial neural network is greater than or equal to the target value. Controls the module to perform bit quantization on another layer among the plurality of layers, and the bit quantization module performs n bits for all weights of the plurality of layers (where n is an integer where n>0). ) is set, and m bits (where m is an integer where m>0) are set for the output data of the plurality of layers.

According to one embodiment of the present disclosure, a bit quantization system for an artificial neural network is provided. This system includes a parameter selection module that selects at least one layer among a plurality of layers constituting the artificial neural network; a bit quantization module that reduces the size of the data representation for the parameter of the selected layer in bits; and an accuracy determination module that determines whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value, and the accuracy determination module determines whether the accuracy of the artificial neural network is greater than or equal to the target value. Controls the module to perform bit quantization on another layer among the plurality of layers, and the bit quantization module performs n bits for the weight and output data of the plurality of layers (where n is n > 0). , an integer) is allocated, but the number of bits allocated to each of the plurality of layers is set differently.

According to one embodiment of the present disclosure, a bit quantization system for an artificial neural network is provided. This system includes a parameter selection module that selects at least one layer among a plurality of layers constituting the artificial neural network; a bit quantization module that reduces the size of the data representation for the parameter of the selected layer in bits; and an accuracy determination module that determines whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value, and the accuracy determination module determines whether the accuracy of the artificial neural network is greater than or equal to the target value. The module is controlled to perform bit quantization on another layer among the plurality of layers, and the bit quantization module individually allocates differently the weights of the plurality of layers and the number of bits of output data.

According to an embodiment of the present disclosure, a bit quantization system for an artificial neural network is provided. This system includes a parameter selection module that selects at least one layer among a plurality of layers constituting the artificial neural network; a bit quantization module that reduces the size of the memory for storing the parameters of the selected layer in bits; and an accuracy determination module that determines whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value, and the accuracy determination module determines whether the accuracy of the artificial neural network is greater than or equal to the target value. The module is controlled to perform bit quantization on another layer among the plurality of layers, and the bit quantization module allocates a different number of bits to each weight used in the plurality of layers.

According to one embodiment of the present disclosure, a bit quantization system for an artificial neural network is provided. This system includes a parameter selection module that selects at least one layer among a plurality of layers constituting the artificial neural network; a bit quantization module that reduces the size of the data representation for the parameter of the selected layer in bits; and an accuracy determination module that determines whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value, and the accuracy determination module determines whether the accuracy of the artificial neural network is greater than or equal to the target value. Controls a module to perform bit quantization on another layer among the plurality of layers, and the bit quantization module individually allocates a different number of bits to specific units of output data output from the plurality of layers. do.

According to one embodiment of the present disclosure, a bit quantization system for an artificial neural network is provided. This system includes a parameter selection module that selects at least one layer among a plurality of layers constituting the artificial neural network; a bit quantization module that reduces the size of the data representation for the parameter of the selected layer in bits; and an accuracy determination module that determines whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value, and the accuracy determination module determines whether the accuracy of the artificial neural network is greater than or equal to the target value. The module is controlled to perform bit quantization on another layer among the plurality of layers, and the bit quantization module allocates different bits to individual values of output data output from the plurality of layers.

According to various embodiments of the present disclosure, overall computational performance can be improved by quantizing the number of bits of data required for operations such as learning or inference in an artificial neural network. In addition, it is possible to reduce the hardware resources required to implement an artificial neural network, reduce power consumption and memory usage, and implement an artificial neural network without deterioration in artificial intelligence accuracy.

The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the scope of Cheongguk.

Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, in which like reference numerals indicate like elements, but are not limited thereto.
FIG. 1 is a diagram showing an example of an artificial neural network that obtains output data for input data using a plurality of layers and a plurality of layer weights according to an embodiment of the present disclosure.
FIGS. 2 and 3 are diagrams for explaining specific implementation examples of the artificial neural network shown in FIG. 1 according to an embodiment of the present disclosure.
Figure 4 is a diagram showing another example of an artificial neural network including a plurality of layers according to an embodiment of the present disclosure.
FIG. 5 is a diagram illustrating input data and a weight kernel used in a convolution operation in a convolution layer according to an embodiment of the present disclosure.
FIG. 6 is a diagram illustrating a procedure for generating a first activation map by performing convolution on input data using a first weight kernel according to an embodiment of the present disclosure.
FIG. 7 is a diagram illustrating a procedure for generating a second activation map by executing convolution on input data using a second weight kernel according to an embodiment of the present disclosure.
Figure 8 is a diagram expressing the calculation process of a convolutional layer according to an embodiment of the present disclosure as a matrix.
Figure 9 is a diagram expressing the calculation process of a fully connected layer according to an embodiment of the present disclosure as a matrix.
Figure 10 is a diagram expressing the bit quantization process of a convolutional layer according to an embodiment of the present disclosure as a matrix.
Figure 11 is a flowchart showing a bit quantization method of an artificial neural network according to an embodiment of the present disclosure.
Figure 12 is a flowchart showing a bit quantization method of an artificial neural network according to another embodiment of the present disclosure.
Figure 13 is a flowchart showing a bit quantization method of an artificial neural network according to another embodiment of the present disclosure.
Figure 14 is a graph showing an example of the amount of calculation for each layer of an artificial neural network according to an embodiment of the present disclosure.
Figure 15 is a graph showing the number of bits per layer of an artificial neural network in which bit quantization has been performed by the forward bit quantization method according to an embodiment of the present disclosure.
FIG. 16 is a graph showing the number of bits per layer of an artificial neural network in which bit quantization is performed using a backward bit quantization method according to an embodiment of the present disclosure.
FIG. 17 is a graph showing the number of bits per layer of an artificial neural network in which bit quantization is performed using a high computational cost layer first bit quantization method according to an embodiment of the present disclosure.
FIG. 18 is a graph showing the number of bits per layer of an artificial neural network in which bit quantization is performed using a low computational cost layer first bit quantization method according to an embodiment of the present disclosure.
FIG. 19 is a diagram illustrating an example of hardware implementation of an artificial neural network according to an embodiment of the present disclosure.
FIG. 20 is a diagram illustrating an example of hardware implementation of an artificial neural network according to another embodiment of the present disclosure.
FIG. 21 is a diagram illustrating an example of hardware implementation of an artificial neural network according to another embodiment of the present disclosure.
FIG. 22 is a diagram illustrating the configuration of a system that performs bit quantization for an artificial neural network according to an embodiment of the present disclosure.

Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the attached drawings. However, in the following description, detailed descriptions of well-known functions or configurations will be omitted if there is a risk of unnecessarily obscuring the gist of the present disclosure.

In the accompanying drawings, identical or corresponding components are given the same reference numerals. Additionally, in the description of the following embodiments, overlapping descriptions of identical or corresponding components may be omitted. However, even if descriptions of components are omitted, it is not intended that such components are not included in any embodiment.

In the present disclosure, “parameter” may mean any one or more of weight data, feature map data, and activation map data of an artificial neural network or each layer constituting an artificial neural network. Additionally, “parameter” may refer to an artificial neural network or each layer constituting an artificial neural network expressed as such data. Additionally, in this disclosure, “bit quantization” may mean an operation or operation that reduces the number of bits of a data representation representing a parameter or group of parameters.

The present disclosure provides various embodiments of a quantization method and system that reduce the data representation size of parameters used in related operations in bits in order to reduce the amount of computation, memory usage, and power consumption of a digital hardware system. In some embodiments, the bit quantization method and system of the present disclosure can reduce the size of parameters used in the calculation of an artificial neural network in units of bits. In general, artificial neural network operations use 32-bit, 16-bit, or 8-bit data structures (e.g., CPU, GPU, memory, cache, buffer, etc.). Accordingly, the quantization method and system of the present disclosure can reduce the size of parameters used in artificial neural network operations to bits other than 32, 16, and 8 bits. Moreover, a specific number of bits can be individually and differently assigned to each parameter or each group of parameters of the artificial neural network.

In some embodiments, the bit quantization method and system of the present disclosure sets n bits (n is an integer where n > 0) for all weights for an artificial neural network model, and sets the output data of each layer to m bits (m can set an integer where m > 0.

In another embodiment, the bit quantization method and system of the present disclosure may allocate n bits to the weight and output data of each layer of the artificial neural network model, where n may be set to a different number for each layer.

In another embodiment, the bit quantization method and system of the present disclosure allocates different bits to the weights and output data of each layer of the artificial neural network model, and also assigns different bits to the weights and output feature map parameters in each layer for each layer. Different numbers of bits can be assigned.

The bit quantization method and system of the present disclosure can be applied to various types of artificial neural networks. For example, when the bit quantization method and system of the present disclosure are applied to a convolutional neural network (CNN), different bits may be individually assigned to the weight kernels used within each layer of the artificial neural network. You can.

In another embodiment, the bit quantization method and system of the present disclosure allocates different bits to each weight used within each layer of a multi-layer artificial neural network model, or individual bits as specific units of output data of each layer. Alternatively, different bits can be assigned to individual values of the output data of each layer.

The bit quantization method and system according to various embodiments of the present disclosure described above may apply any one of the embodiments described above to an artificial neural network model, but is not limited thereto, and may apply one or more of these embodiments. It can also be combined and applied to an artificial neural network model.

FIG. 1 is a diagram showing an example of an artificial neural network 100 that obtains output data for input data using a plurality of layers and a plurality of layer weights according to an embodiment of the present disclosure.

In general, an artificial neural network with a multi-layer structure, such as the artificial neural network 100, is a statistical learning algorithm implemented based on the structure of a biological neural network or a structure for executing the algorithm in machine learning technology and cognitive science. Includes. In other words, in the artificial neural network 100, nodes, which are artificial neurons that form a network by combining synapses, as in a biological neural network, repeatedly adjust the weights of the synapses to produce the correct output and inferred output corresponding to a specific input. By learning to reduce the error between the two, a machine learning model with problem-solving capabilities can be created.

In one example, the artificial neural network 100 may be implemented as a multilayer perceptron (MLP) consisting of layers containing one or more nodes and connections between them. However, the artificial neural network 100 according to this embodiment is not limited to the MLP structure, and can be implemented using one of various artificial neural network structures having a multi-layer structure.

As shown in FIG. 1, when input data is input from the outside, the artificial neural network 100 passes through a plurality of layers (110_1, 110_2, ..., 110_N), each composed of one or more nodes, to generate a layer corresponding to the input data. It is configured to output output data.

In general, the learning method of the artificial neural network 100 includes a supervised learning method that learns to optimize problem solving by inputting a teacher signal (correct answer), and an unsupervised learning method that does not require a teacher signal. Learning) method, and semi-supervised learning method that uses both supervised learning and unsupervised learning. The artificial neural network 100 shown in FIG. 1 uses at least one of a supervised learning method, an unsupervised learning method, and a semi-supervised learning method according to the user's selection. Using this, the artificial neural network 100 that generates output data can be trained.

FIGS. 2 and 3 are diagrams for explaining specific implementation examples of the artificial neural network 100 shown in FIG. 1 according to an embodiment of the present disclosure.

Referring to FIG. 2, the artificial neural network 200 has an input node ( , ... , ), an output node that outputs output data corresponding to the input data 210 ( , ... , ), a hidden node located between the input node and the output node, and a number of parameters may be included. input node ( , ... , ) is a node constituting the input layer 220, which receives input data 210 (e.g., image) from the outside, and an output node ( , ... , ) is a node that constitutes the output layer 240 and can output output data to the outside. The hidden node located between the input node and the output node is a node that constitutes the hidden layer 230, and can connect the output data of the input node to the input data of the output node. As shown in FIG. 2, each node of the input layer 220 may be fully or incompletely connected to each output node of the output layer 240. Additionally, the input node may serve to receive input data from the outside and transmit it to the hidden node. At this time, calculations on data can be performed at the hidden node and output node, and calculations can be performed by multiplying the received input data by parameters (or weights). When the calculation of each node is completed, all calculation results can be added together and output data can be output using a preset activation function.

Hidden nodes and output nodes ( , ... , ) has an activation function. Activation functions include step function, sign function, linear function, logistic sigmoid function, hyper tangent function, ReLU function, and softmax ( softmax) function. The activation function can be appropriately determined by a person skilled in the art according to the learning method of the artificial neural network.

The artificial neural network 200 performs machine learning by repeatedly updating (or modifying) weight values to appropriate values. Representative methods of machine learning by the artificial neural network 200 include supervised learning and unsupervised learning.

Supervised learning involves updating weight values so that the output data obtained by inputting the input data into the neural network is similar to the target data, in a state where the target output data that a random neural network wants to calculate for the input data is clearly determined. It's a learning method. The multi-layered artificial neural network 200 of FIG. 2 can be created based on supervised learning.

Referring to FIG. 3, another example of a multi-layer artificial neural network is a convolutional neural network (CNN) 300, which is a type of deep neural network (DNN). A convolutional neural network (CNN) is a neural network composed of one or several convolutional layers, a pooling layer, and a fully connected layer. Convolutional neural network (CNN) has a structure suitable for learning two-dimensional data, and can be learned through the backpropagation algorithm. It is one of the representative models of DNN that is widely used in various application fields such as object classification and object detection in images.

Here, it should be noted that the multi-layered artificial neural network of the present invention is not limited to the artificial neural network shown in Figures 2 and 3, and that the learned model can be obtained by machine learning other types of data in various other artificial neural networks. do.

Figure 4 is a diagram showing another example of an artificial neural network including a plurality of layers according to an embodiment of the present disclosure. The artificial neural network 400 shown in FIG. 1 includes a plurality of convolution layers (CONV) 420, a plurality of subsampling layers (SUBS) 430, and a plurality of subsampling layers (SUBS) 430 disclosed in FIG. 3. It is a convolutional neural network (CNN) including a fully-connected layer (FC) 440.

The CONV 420 of the CNN 400 generates a feature map by applying a convolutional weight kernel to the input data 410. Here, CONV 420 may serve as a type of template that extracts features from high-dimensional input data (for example, images or videos). Specifically, one convolution can be applied repeatedly several times while changing the position of a portion of the input data 410 to extract features for the entire input data 410. Additionally, the SUBS 430 serves to reduce the spatial resolution of the feature map generated by the CONV 420. Subsampling functions to reduce the dimension of input data (eg, feature map), and through this, the complexity of the analysis problem of the input data 410 can be reduced. The SUBS 430 may use a max pooling operator that takes the maximum value of the values of the portion of the feature map or an average pooling operator that takes the average value. Such SUBS 430 not only reduces the dimension of the feature map through a pooling operation, but also has the effect of making the feature map robust against shift and distortion. Lastly, the FC 440 can perform the function of classifying input data based on the feature map.

CNN (400) can execute various configurations and functions depending on the number of layers or type of operator of CONV (420), SUBS (430), and FC (440). For example, the CNN 400 may include any one of various CNN configurations such as AlexNet, VGGNet, LeNet, ResNet, etc., but is not limited thereto.

When image data is input as input data 410, the CONV 420 of the CNN 400 having the configuration described above can apply weights to the input data 410 and generate a feature map through a convolution operation. , At this time, the group of weights used may be referred to as a weight kernel. The weight kernel is a three-dimensional matrix of n The number is an integer of 1 or more), and the input data 410 can be traversed at specified intervals and a feature map can be generated through a convolution operation. At this time, if the input data 410 is a color image with a plurality of channels (for example, three channels of RGB), the weight kernel traverses each channel of the input data 410, calculates the convolution, and then calculates the convolution for each channel. A feature map can be created.

Figure 5 is a diagram showing input data of a convolution layer and a weight kernel used in a convolution operation according to an embodiment of the present disclosure.

As shown, the input data 510 may be an image or video displayed as a two-dimensional matrix consisting of rows 530 of a specific size and columns 540 of a specific size. As previously described, the input data 510 may have a plurality of channels 550, where the channels 550 may represent the number of color components of the input data image. Meanwhile, the weight kernel 520 may be a weight kernel used in convolution to extract features of a certain portion of the input data 510 while scanning it. The weight kernel 520 may be configured to have rows 560 of a specific size, columns 570 of a specific size, and a specific number of channels 580, similar to the input data image. In general, the sizes of the rows 560 and columns 570 of the weight kernel 520 are set to be the same, and the number of channels 580 may be the same as the number of channels 550 of the input data image.

FIG. 6 is a diagram illustrating a procedure for generating a first activation map by executing convolution on input data using a first kernel according to an embodiment of the present disclosure.

The first weight kernel 610 may be a weight kernel representing the first channel of the weight kernel 620 of FIG. 2. The first weight kernel 610 can ultimately generate the first activation map 630 by traversing the input data at designated intervals and performing convolution. Convolution, when applying the first weight kernel 610 to a portion of the input data 510, multiplies the input data values at a specific location of that portion and the values at the corresponding location of the weight kernel, and then all the values generated are It is executed in addition. Through this convolution process, a first result value 620 is generated, and each time the first weight kernel 610 traverses the input data 510, the result values of this convolution are generated to form a feature map. . Each component value of the feature map is converted into the first activation map 630 through the activation function of the convolution layer.

FIG. 7 is a diagram illustrating a procedure for generating a second activation map by executing convolution on input data using a second weight kernel according to an embodiment of the present disclosure.

As shown in FIG. 6, the first activation map 620 is generated by performing convolution on the input data 510 using the first weight kernel 610, and then the second weight is applied as shown in FIG. 7. The second activation map 730 can be generated by performing convolution on the input data 510 using the kernel 710.

The second weight kernel 710 may be a weight kernel representing the second channel of the weight kernel 520 of FIG. 5. The second weight kernel 710 can ultimately generate the second activation map 730 by traversing the input data at specified intervals and performing convolution. Similar to FIG. 6, convolution is performed by multiplying the input data values at a specific location of the portion and the values at the corresponding location of the weight kernel when the second weight kernel 710 is applied to a portion of the input data 510. It is executed by adding up all the generated values. Through this convolution process, a second result value 720 is generated, and each time the second weight kernel 710 traverses the input data 510, the result values of this convolution are generated to form a feature map. . Each component value of the feature map is converted into a second activation map 730 through the activation function of the convolution layer.

FIG. 8 is a diagram expressing the calculation process of a convolutional layer in a matrix when an input feature map has one channel according to an embodiment of the present disclosure.

The convolution layer 420 shown in FIG. 8 may correspond to the CONV 420 shown in FIG. 4. In Figure 8, the input data 810 input to the convolution layer 420 is displayed as a two-dimensional matrix with a size of 6 x 6, and the weight kernel 814 is a two-dimensional matrix with a size of 3 x 3. displayed. However, the size of the input data 810 and the weight kernel 814 of the convolution layer 420 are not limited to this, and may vary depending on the performance and requirements of the artificial neural network including the convolution layer 420. can be changed.

As shown, when input data 810 is input to the convolution layer 420, the weight kernel 814 traverses the input data 810 at predetermined intervals (e.g., 1) and calculates the input data ( Elementwise multiplication can be performed by multiplying the values at the same position of the weight kernel 810) and the weight kernel 814. The weight kernel 814 iterates over the input data 810 at regular intervals and sums (816) the values obtained through multiple multiplication.

Specifically, the weight kernel 814 assigns the value of the multiple product (e.g., “3”) calculated at a specific position 820 of the input data 810 to the corresponding element 824 of the feature map 818. do. Next, the weight kernel 814 assigns the value of the multiple product (e.g., “1”) calculated at the next position 822 of the input data 810 to the corresponding element 826 of the feature map 818. do. In this way, when the weight kernel 814 traverses the input data 810 and allocates all of the multi-product values calculated to the feature map 818, the feature map 818 of 4 x 4 size is completed. At this time, if the input data 810 consists of, for example, three channels (R channel, G channel, B channel), the same weight kernel or a different channel for each channel is used to compare the data for each channel of the input data 810. Feature maps for each channel can be generated through convolution that traverses and performs multiple multiplication (812) and sum (816).

Referring again to FIG. 4, CONV 420 applies an activation function to the feature map generated according to the method described with reference to FIGS. 25 to 8 to generate an activation map that is the final output result of the convolution layer. can be created. Here, the activation function is one of various activation functions such as a sigmoid function ((sigmoid function), radial basis function (RBF), rectified linear unit (ReLU), or a modified function among them. It could be this or another function.

Meanwhile, the SUBS 430 receives the activation map, which is the output data of the CONV 420, as input data. The SUBS 430 performs a function of reducing the size of the activation map or emphasizing specific data. When SUBS 430 uses max pooling, it selects and outputs the maximum value within a specific area of the activation map. In this way, noise in input data can be removed through the pooling process of the SUBS 430, and the size of the data can be reduced.

Additionally, FC 440 may receive output data from SUBS 430 and generate final output data 450. The activation map extracted from the SUBS 430 is one-dimensionally flattened to be input to the fully connected layer 440.

Figure 9 is a diagram expressing the calculation process of a fully connected layer according to an embodiment of the present disclosure as a matrix.

The fully connected layer 440 shown in FIG. 9 may correspond to the FC 440 in FIG. 4. As described above, the activation map extracted from the max pooling layer 430 can be flattened into one dimension to be input to the fully connected layer 440. The one-dimensional flattened activation map may be received as input data 910 in the fully connected layer 440. In the fully connected layer 440, multiple multiplication 912 of the input data 910 and the weight kernel 914 can be performed using the one-dimensional weight kernel 914. The result of multiple products of the input data 910 and the weight kernel 914 may be summed 916 and output as output data 918. At this time, the output data 918 may represent an inference value for the input data 410 input to the CNN 400.

The CNN 400 having the configuration described above receives input data of a two-dimensional or one-dimensional matrix for each of a plurality of layers, and performs learning and inference through complex operations such as multiple multiplication and summation of weight kernels on the input data. Run the process. Therefore, depending on the number of layers constituting the CNN 400 or the complexity of the operation, the resources (for example, the number of operators or the amount of memory) required for learning and inferring data may increase significantly. Therefore, in order to reduce the amount of computation and memory of an artificial neural network with multiple layers, such as the CNN 400, bit quantization can be performed on input and output data used for each layer. In one embodiment, bit quantization of the CNN 400 with multiple layers may be performed on the CONV 420 and FC 440, which require a large amount of computation and memory.

Figure 10 is a diagram expressing the bit quantization process of a convolutional layer according to an embodiment of the present disclosure as a matrix.

Bit quantization performed in the convolutional layer includes weight or weight kernel quantization 1028, which reduces the number of bits of the value of each element of the weight kernel used in the convolution operation, and/or of the value of each element of the feature map or activation map. It may include feature map quantization or activation map quantization 1030, which reduces the number of bits.

The bit quantization process of the convolution layer according to one embodiment may be performed as follows. Before performing convolution by applying the weight kernel 1014 to the input data 1010 of the convolution layer, a quantization 716 process is performed on the weight kernel 1014 to generate a quantized weight kernel 1018. In addition, the quantized weight kernel 1018 is applied to the input data 1010, multiple multiplication (1012) and summation (1020) are performed, the convolution values are output to generate a feature map, and then activated through an activation function. A map 1022 can be created. Next, the final quantized activation map 1026 can be generated through quantization 1024 on the activation map.

In the bit quantization process of the convolutional layer described above, weight kernel quantization 1028 can be performed using the following equation.

here, represents the weight values to be quantized (e.g., real weights and each weight in the weight kernel), represents the number of bits to be quantized, Is It represents the result quantized by k bits. That is, according to the above formula, first For, a predetermined binary number Multiply by , The number of digits is increased by k bits (hereinafter referred to as “first value”). Next, by performing a rounding or truncation operation on the first value, The digits after the decimal point are removed (hereinafter referred to as the "second value"). The second value is binary By dividing by and reducing the number of digits by k bits again, the element value of the final quantized weight kernel can be calculated. This weight or weight kernel quantization 1028 is repeatedly performed on all element values of the weight or weight kernel 1014 to generate quantized weight values 1018.

Meanwhile, feature map or activation map quantization 1030 can be performed using the following equation.

In feature map or activation map quantization (1030), the same formula as the weight or weight kernel quantization (1028) method can be used. However, in the feature map or activation map quantization 1030, the value of each element of the feature map or activation map 1022 ( ) (e.g., the coefficient of a real number), before quantization is applied, clipping is applied to normalize the value of each element of the feature map or activation map 1022 to a value between 0 and 1. can do.

Next, the normalized For, a predetermined binary number Multiply by The number of digits is increased by k bits (“first value”). Next, by performing a rounding or truncation operation on the first value, The decimal digits of are removed (“second value”). The second value is binary By dividing by and reducing the number of digits by k bits again, the element value of the final quantized feature map or activation map 1026 can be calculated. This quantization 1030 of the feature map or activation map is repeatedly performed on all element values of the feature map or activation map 1022, thereby generating a quantized feature map or activation map 1026.

Through the weight or weight kernel quantization (1028) and the feature map or activation map quantization (1030) described above, the memory size and amount of calculation required for the convolution operation of the convolution layer 420 of the convolutional neural network can be reduced in bits. You can.

Figure 11 is a flowchart showing a bit quantization method of an artificial neural network according to an embodiment of the present disclosure. This embodiment is an example in which the unit of a data group that can be quantized in an artificial neural network is assumed to be all parameters belonging to each layer constituting the artificial neural network.

As shown, the bit quantization method 1100 of an artificial neural network may begin with selecting at least one layer among a plurality of layers included in the artificial neural network (S1110). Which layer to select from among the plurality of layers included in the artificial neural network may be determined based on the effect of the selected layer on the overall performance or amount of computation (or amount of memory) of the artificial neural network. In one embodiment, in the multi-layered artificial neural network described above with reference to FIGS. 1 to 3, a layer that has a large impact on the overall performance or computational amount of the artificial neural network may be arbitrarily selected. In addition, in the case of the convolutional artificial neural network (CNN) 400 described with reference to FIGS. 4 to 10, the convolutional layer 420 and/or the fully connected layer 440 are the overall performance or computational amount of the CNN 400. Because it has a large influence on the etc., at least one layer among these layers 420 and 440 may be selected.

The method of selecting at least one of the plurality of layers included in the artificial neural network may be determined depending on the effect of the selected layer on the overall performance or computational amount of the artificial neural network, but is not limited to this and may include one of various methods. You can. For example, the selection of at least one layer among the plurality of layers included in the artificial neural network is (i) sequentially from the first layer where input data is received to subsequent layers according to the arrangement order of the plurality of layers constituting the artificial neural network. (ii) a method of sequentially selecting from the last layer where the final output data is generated to the previous layer according to the arrangement order of the plurality of layers constituting the artificial neural network, (iii) the plurality of layers constituting the artificial neural network It may be executed by selecting the layer with the highest computational amount among the layers, or (iv) selecting the layer with the lowest computational amount among the plurality of layers that make up the artificial neural network.

Once the selection of the artificial neural network layer is completed in step S1110, the step S1120 may proceed to reduce the size of the data representation for the parameter (eg, weight) of the selected layer in bit units.

In one embodiment, when the size of the weight or output data among the parameters of the selected layer is reduced in bits, the weight kernel quantization 1028 and activation map quantization 1024 described with reference to FIGS. 4 to 10 may be performed. there is. For example, weight kernel quantization 1028 can be calculated by the following equation.

here, represents the element value of the weight kernel to be quantized (e.g., a real weight kernel coefficient), represents the number of bits to be quantized, Is It represents the result quantized by k bits. That is, according to the above formula, first For, a predetermined binary number Multiply by , The number of digits is increased by k bits (“first value”). Next, by performing a rounding or truncation operation on the first value, The decimal digits of are removed (“second value”). The second value is binary By dividing by and reducing the number of digits by k bits again, the element value of the final quantized weight kernel can be calculated. This weight kernel quantization (1028) is repeatedly performed on all element values of the weight kernel (1014), thereby generating the quantization weight kernel (1018).

Meanwhile, activation map quantization 1030 can be performed using the following equation.

In activation map quantization 1030, the value of each element of the activation map 1022 ( ) (e.g., the coefficient of a real number), before quantization is applied, a process of normalizing the value of each element of the activation map 1022 to a value between 0 and 1 can be added by applying clipping. . Next, the normalized For, a predetermined binary number Multiply by The number of digits is increased by k bits (“first value”). Next, by performing a rounding or truncation operation on the first value, The decimal digits of are removed (“second value”). The second value is binary By dividing by and reducing the number of digits by k bits again, the element value of the final quantized activation map 1026 can be calculated. This quantization 1030 of the activation map is repeatedly performed on all element values of the activation map 1022, thereby generating the quantized activation map 1026.

In the above-described embodiments, an example of reducing the number of bits of the weight value or activation map data was described in order to reduce the size of the data representation for the parameter of the selected layer in the artificial neural network, but the bit quantization method of the present disclosure is It is not limited to this. In another embodiment, different bits may be allocated to data in an interruption stage that exists between various calculation stages for various data included in a selected layer in an artificial neural network. Accordingly, in order to reduce the size of the memory (e.g., buffer, register, or cache) in which each data is stored when implementing the artificial neural network in hardware, the number of bits of each data stored in the memory is reduced and the number of bits in the corresponding memory is reduced. The number of bits can also be reduced. In another embodiment, the size of the data bits of the data path through which data of the selected layer in the artificial neural network is transmitted may be reduced in bit units.

After executing step S1120, a step S1130 may be performed to determine whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value. After reducing the data representation size of the parameters of the selected layer in the artificial neural network in bits, if the accuracy of the output result of the artificial neural network (e.g., learning result or inference result of the artificial neural network) is greater than a predetermined target value, additional It can be expected that the overall performance of the artificial neural network can be maintained even if the bits of the data are reduced.

Accordingly, if it is determined in step S1130 that the accuracy of the artificial neural network is greater than the target value, the process proceeds to step S1120 and the data representation size of the selected layer can be further reduced in bit units. Additionally, it may be determined again whether the accuracy of the output result of the corresponding artificial neural network (e.g., learning result or inference result of the artificial neural network) is greater than or equal to a predetermined target value (step S1130).

In step S1130, if the accuracy of the artificial neural network is not greater than the target value, it may be determined that the accuracy of the artificial neural network has been degraded due to the currently performed bit quantization. Therefore, in this case, the minimum number of bits that satisfied the accuracy target value in the bit quantization performed immediately before can be determined as the final number of bits for the parameter of the selected layer (step S1140).

Next, it is determined whether bit quantization for all layers of the artificial neural network has been completed (step S1150). At this stage, when it is determined that bit quantization for all layers of the artificial neural network is complete, the entire process is terminated. On the other hand, if there are still layers among the layers of the artificial neural network that have not yet been bit quantized, step S1110 is executed to perform bit quantization for the corresponding layer.

Here, the method of selecting another layer from among the plurality of layers included in the artificial neural network in step S1110 is (i) selecting the next layer of the previously selected layer according to the arrangement order of the plurality of layers constituting the artificial neural network. A method of sequentially selecting (“forward bit quantization”), (ii) a method of selecting the previous layer of the previously selected layer in the reverse direction according to the arrangement order of the plurality of layers constituting the artificial neural network (“forward bit quantization”) Quantization", backward bit quantization), (iii) a method of selecting the layer with the highest computational cost after the previously selected layer according to the order of computational amount among the plurality of layers that make up the artificial neural network ("high computational bit quantization", high computational cost bit quantization), or (iv) a method of selecting the layer with the smallest computational cost following the previously selected layer in the order of computational amount among the plurality of layers that make up the artificial neural network ("low computational cost bit quantization", low computational cost bit quantization) It may be executed depending on .

In one embodiment, the accuracy of an artificial neural network refers to the fact that the artificial neural network learns how to solve a given problem (for example, recognition of an object included in an image as input data) and then presents a solution to the problem in the inference stage. It can mean probability. Additionally, the target value used in the bit quantization method described above may represent the minimum accuracy to be maintained after bit quantization of the artificial neural network. For example, assuming that the target value is 90% accuracy, even after reducing the parameters of the selected layer in bits by bit quantization, if the accuracy of the artificial neural network is more than 90%, additional bit quantization can be performed. For example, after performing the first bit quantization, if the accuracy of the artificial neural network is measured to be 94%, additional bit quantization can be performed. After execution of the second bit quantization, if the accuracy of the artificial neural network is measured to be 88%, the result of the currently performed bit quantization is ignored, and the number of bits determined by the first bit quantization (i.e., the number of bits to represent the corresponding data) can be confirmed as the final bit quantization result.

In one embodiment, according to a computational cost bit quantization method, in an artificial neural network including a plurality of layers, when selecting a layer to perform bit quantization among a plurality of layers based on the computational cost, the computational cost of each layer can be determined as follows. In other words, if one addition operation performs the addition of n bits and m bits in a specific layer of the artificial neural network, the amount of calculation for that operation is calculated as (n+m)/2. Additionally, when a specific layer of the artificial neural network performs multiplication of n bits and m bits, the amount of computation for the corresponding operation can be calculated as n x m. Therefore, the amount of computation of a specific layer of an artificial neural network can be the result of adding up the amount of computations of all additions and multiplications performed by that layer.

In addition, according to the computational cost bit quantization method, the method of performing bit quantization by selecting a layer based on the computational cost among a plurality of layers in an artificial neural network is not limited to the one shown in FIG. 11, and can be used in various ways. Transformation is possible.

In another embodiment, the bit quantization of parameters for each layer in the embodiment shown in FIG. 11 may be performed separately for each weight and activation map. For example, first, quantization is performed on the weight of the selected layer, and as a result, the weight has n bits. Separately, bit quantization may be performed on the output activation data of the selected layer to determine the number of bits representing the activation map data as m bits. Alternatively, quantization may be performed while assigning the same bits to the weight and activation map data of the corresponding layer, and as a result, both the weight and activation map data may be expressed with the same n bits.

Figure 12 is a flowchart showing a bit quantization method of an artificial neural network according to another embodiment of the present disclosure.

As shown, the bit quantization method 1200 of an artificial neural network may begin with a step (S1210) of selecting a layer with the highest calculation amount among a plurality of layers included in the artificial neural network.

Once the selection of the artificial neural network layer is completed in step S1210, the step S1220 may proceed to reduce the size of the data representation for the parameters of the selected layer in bits. In one embodiment, when reducing the size of the data of the selected layer in bits, the weight kernel quantization 1028 and activation map quantization 1024 described with reference to FIGS. 4 to 10 may be performed.

After executing step S1220, a step S1230 can be performed to determine whether the accuracy of the artificial neural network reflecting the bit quantization results so far is greater than or equal to a predetermined target value. If it is determined in step S1230 that the accuracy of the artificial neural network is greater than the target value, set the data size of the corresponding layer to the current bit quantization result, proceed to step S1210, and repeat steps S1210 to S1230. You can. That is, proceeding to step S1210, the calculation amount is recalculated for all layers in the artificial neural network, and based on this, the layer with the highest calculation amount is selected again.

In step S1230, if the accuracy of the artificial neural network is not higher than the target value, bit reduction quantization for the currently selected layer is canceled, and the layer is excluded from the layers that can be selected in the layer selection step S1210. Then, the layer with the next highest calculation amount after that layer can be selected (step S1240). Next, the size of the data of the selected layer can be reduced in bit units (step S1250).

In step S1260, it is determined whether the accuracy of the artificial neural network reflecting the bit quantization results so far is greater than or equal to the target value. If the accuracy of the artificial neural network is not higher than the target value, it is determined whether bit quantization for all layers of the artificial neural network has been completed (S1270). If it is determined in step S1270 that bit quantization for all layers of the artificial neural network is completed, the entire bit quantization procedure is terminated. On the other hand, if it is determined in step S1270 that bit quantization for all layers of the artificial neural network has not been completed, the process may proceed to step S1240.

If it is determined in step S1260 that the accuracy of the artificial neural network is greater than the target value, the process can proceed to step S1220 and further procedures.

Figure 13 is a flowchart showing a bit quantization method of an artificial neural network with a plurality of layers according to another embodiment of the present disclosure.

As shown, the bit quantization method 1300 of an artificial neural network having a plurality of layers includes steps (S1310 to S1350) of searching for accuracy change points for each of all layers included in the artificial neural network. The method 1300 begins with a step (S1310) of initially fixing the bit size of data of all layers included in the artificial neural network to the maximum and selecting one layer for which no search for accuracy change points has been performed.

Once the selection of a layer of a random artificial neural network is completed in step S1310, the process may proceed to step S1320 of reducing the size of the data of the selected layer in bits. In one embodiment, when reducing the size of the data of the selected layer in bits, the weight kernel quantization 1028 and activation map quantization 1024 described with reference to FIGS. 4 to 10 may be performed.

After executing step S1320, a step S1330 may be performed to determine whether the accuracy of the artificial neural network reflecting the bit quantization results so far for the selected layer is greater than or equal to a predetermined target value. If it is determined in step S1330 that the accuracy of the artificial neural network is greater than or equal to the target value, the process proceeds to step S1320 to perform additional bit reduction quantization for the currently selected layer.

In step S1330, if the accuracy of the artificial neural network is not higher than the target value, the number of data bits of the currently selected layer is set to the minimum number of bits that most recently satisfied the target value. Afterwards, it is determined whether the search for accuracy change points for all layers of the artificial neural network has been completed (S1340). In this step, if the search for accuracy change points for all layers is not completed, the process may proceed to step S1310. In step S1310, the bit size of the data of all layers included in the artificial neural network is the maximum, and another layer for which performance change points have not been searched is selected.

If, in step S1340, it is determined that the search for accuracy change points for all layers of the artificial neural network is completed, the bit quantization result corresponding to the accuracy change point for each layer of the artificial neural network can be reflected in the artificial neural network. (S1350). In one embodiment, in step S1350, the accuracy change point of each layer of the artificial neural network determined according to the steps S1310 to S1340 described above (e.g., the point where the accuracy of the artificial neural network is deteriorated in each layer) point) Set the layer to the bit size of the previous data.

In another embodiment, in step S1350, the corresponding layer is set to be larger than the size of the resources required for calculating the parameter immediately before the accuracy change point of each layer of the artificial neural network determined according to the steps S1310 to S1340 described above. do. For example, the number of bits of the parameters of each layer of the artificial neural network can be set to be 2 bits larger than the number of bits immediately before the accuracy change point. Next, the bit quantization method is executed on the artificial neural network having the data size of each layer set in step 1350 (S1360). The bit quantization method performed in step S1360 may include, for example, the method shown in FIG. 11 or FIG. 12.

The bit quantization method of the artificial neural network according to the various embodiments described above is not limited to being executed on the weight kernel and feature map (or activation map) of each of the plurality of layers of the artificial neural network. In one embodiment, the bit quantization method of the present disclosure is first executed on the weight kernel (or weights) of all layers of the artificial neural network, and then bit again on the feature maps of all layers of the artificial neural network in which such weight kernel quantization is reflected. Quantization may also be performed. In another embodiment, bit quantization may be performed first on the feature maps of all layers of the artificial neural network, and then bit quantization may be performed again on the kernels of all layers of the artificial neural network in which the feature map quantization is reflected.

Additionally, the bit quantization method of the artificial neural network of the present disclosure is not limited to applying the same level of bit quantization to the weight kernels of each layer of the artificial neural network. In one embodiment, the bit quantization method of the present disclosure may perform bit quantization on a weight kernel basis of each layer of the artificial neural network, or may perform individual bit quantization so that each weight unit that is an element of each weight kernel can have a different bit. Quantization can also be performed.

Hereinafter, examples of execution results of the bit quantization method of an artificial neural network according to various embodiments of the present disclosure will be described with reference to the drawings.

Figure 14 is a graph showing an example of the amount of calculation for each layer of an artificial neural network according to an embodiment of the present disclosure. The artificial neural network shown in Figure 14 is an example of a convolutional artificial neural network of the VGG-16 model containing 16 layers, and each layer of this artificial neural network has a different computational amount.

For example, the 2nd layer, 4th layer, 6th layer, 7th layer, 9th layer, and 10th layer have the highest computational cost, so they follow the high computational cost bit quantization method. In this case, bit quantization may be applied first. Additionally, after bit quantization is performed for the second, fourth, sixth, seventh, ninth, and tenth layers, bit quantization may be performed for the fourteenth layer, which has the next highest calculation amount.

FIG. 15 is a graph showing the number of bits per layer of an artificial neural network in which bit quantization is performed using a forward bit quantization method according to an embodiment of the present disclosure.

As previously explained, forward quantization is a method of sequentially performing bit quantization starting from the frontmost layer (e.g., from the layer where input data is first received) based on the arrangement order of the plurality of layers included in the artificial neural network. am. Figure 15 shows the number of bits for each layer after applying forward quantization to the artificial neural network of the VGG-16 model shown in Figure 14, and the reduction rate of the computational amount of the artificial neural network due to forward quantization. For example, when performing addition of n bits and m bits, the computational amount of the operation is calculated as (n+m)/2. Additionally, when multiplication of n bits and m bits is performed, the amount of computation for the corresponding operation can be calculated as n x m. Therefore, the total computational amount of the artificial neural network may be the result of adding up all the computational amounts of addition and multiplication performed by the artificial neural network.

As shown, when bit quantization was performed on the artificial neural network of the VGG-16 model using forward quantization, the number of bits in the layers arranged in front of the artificial neural network was relatively reduced significantly, and the number of bits in the layers arranged behind the artificial neural network was relatively reduced. The number of bits decreased slightly. For example, the number of bits of the first layer of the artificial neural network decreased to 12 bits, the number of bits of the second layer and the third layer decreased to 9 bits each, while the number of bits of the 16th layer decreased to 13 bits. , the number of bits in the 15th layer was reduced to only 15 bits. When forward quantization was applied sequentially from the first to the 16th layer of the artificial neural network, the reduction rate of the overall computational amount of the artificial neural network was calculated to be 56%.

FIG. 16 is a graph showing the number of bits per layer of an artificial neural network in which bit quantization is performed using a backward bit quantization method according to an embodiment of the present disclosure.

Reverse quantization is a method of sequentially performing bit quantization starting from the rearmost layer (for example, from the layer where output data is finally output) based on the arrangement order of the plurality of layers included in the artificial neural network. Figure 16 shows the number of bits for each layer after applying reverse quantization to the artificial neural network of the VGG-16 model shown in Figure 14, and the reduction rate of the computational amount of the artificial neural network due to reverse quantization.

As shown, when bit quantization is performed on the artificial neural network of the VGG-16 model using reverse quantization, the number of bits in the layers arranged behind the artificial neural network is relatively reduced, while the number of bits in the layers arranged in front of the artificial neural network is reduced. The number of bits decreased slightly. For example, the number of bits of the first layer, second layer, and third layer each decreased to 15 bits, the number of bits of the fourth layer decreased to 14 bits, while the number of bits of the 16th layer decreased to 9 bits. And the number of bits in the 15th layer was reduced to 15 bits. When reverse quantization was applied sequentially from the first to the 16th layer of the artificial neural network, the reduction rate of the overall computational amount of the artificial neural network was calculated to be 43.05%.

FIG. 17 is a graph showing the number of bits per layer of an artificial neural network in which bit quantization is performed using a high computational cost layer first bit quantization method according to an embodiment of the present disclosure.

High computation layer priority quantization (or high computation amount quantization) is a method of sequentially performing bit quantization starting from the layer with the highest computation amount among the plurality of layers included in the artificial neural network. Figure 17 shows the number of bits for each layer after applying high calculation amount quantization to the artificial neural network of the VGG-16 model shown in Figure 14, and the reduction rate of the calculation amount of the artificial neural network due to high calculation amount quantization.

As shown, when bit quantization was performed on the artificial neural network of the VGG-16 model using high-computation quantization, the number of bits of the layers with high calculation amount among the plurality of layers of the artificial neural network decreased relatively significantly. For example, the number of bits in the second layer and the tenth layer decreased to 5 bits and 6 bits, respectively, while the number of bits in the first layer decreased to 14 bits. When high-computation quantization was applied to the layers of the artificial neural network in order of computation amount, the reduction rate of the overall computation amount of the artificial neural network was calculated to be 70.70%.

FIG. 18 is a graph showing the number of bits per layer of an artificial neural network in which bit quantization is performed using a low computational cost layer priority quantization method according to an embodiment of the present disclosure.

Low computation layer priority quantization (or low computation amount quantization) is a method of sequentially performing bit quantization starting from the layer with the lowest computation amount among a plurality of layers included in an artificial neural network. Figure 18 shows the number of bits for each layer after applying low-computation quantization to the artificial neural network of the VGG-16 model shown in Figure 14, and the reduction rate of the calculation amount of the artificial neural network due to low-computation quantization.

As shown, even when bit quantization was performed on the artificial neural network of the VGG-16 model using low-computation quantization, the number of bits in the layers with high calculation amount among the plurality of layers of the artificial neural network decreased relatively significantly. For example, the number of bits of the 6th layer and the 7th layer decreased to 6 bits and 5 bits, respectively, while the number of bits of the first layer decreased to 13 bits. When low-computation quantization was applied to the layers of the artificial neural network in order of computation amount, the reduction rate of the computation amount of the entire artificial neural network was calculated to be 49.11%.

Hereinafter, hardware implementation examples of an artificial neural network to which bit quantization is applied according to various embodiments of the present disclosure described above will be described in detail. When implementing a convolutional artificial neural network including a plurality of layers in hardware, the weight kernel may be arranged outside and/or inside a processing unit for executing convolution of the convolutional layer.

In one embodiment, the weight kernel may be stored in a memory (e.g., register, buffer, cache, etc.) separate from the processing unit for executing the convolution of the convolution layer. In this case, after applying bit quantization to the weight kernel to reduce the number of bits of the element values of the weight kernel, the size of the memory can be determined according to the number of bits of the weight kernel. In addition, the bit width of a multiplier or adder arranged in a processing unit that receives the element values of the weight kernel and the element values of the input feature map stored in memory and performs multiplication and/or addition operations. It can also be designed according to the number of bits resulting from bit quantization.

In another embodiment, the weight kernel may be implemented in a hard-wired form within a processing unit for executing convolution of the convolution layer. In this case, after applying bit quantization to the weight kernel to reduce the number of bits of the element values of the weight kernel, a hardwire representing each element value of the weight kernel according to the number of bits of the weight kernel can be implemented in the processing unit. In addition, the bit size of the multiplier or adder arranged in the processing unit that receives the element values of the hardwired weight kernel and the element values of the input feature map and performs multiplication and/or addition operations also depends on the number of bits according to the result of bit quantization. It can be designed accordingly.

19 to 21 described below are diagrams showing an example of hardware implementation of an artificial neural network including a plurality of layers according to another embodiment of the present disclosure. The bit quantization method and system of an artificial neural network including a plurality of layers of the present disclosure can be applied to any ANN (Artificial neural network) operation system such as CPU, GPU, FPGA, ASIC, etc. to determine the amount of calculation required, the bit size of the operator, Memory may be reduced. In addition, in this example, the embodiment was shown based on an integer, but it can also be performed with a floating point operation.

FIG. 19 is a diagram illustrating an example of hardware implementation of an artificial neural network according to an embodiment of the present disclosure. The depicted artificial neural network represents an example of hardware implementation of the convolution processing device 1900 of the convolutional layer of the convolutional artificial neural network. Here, the convolution layer is explained assuming that convolution is performed by applying a weight kernel of size 3x3x3 to a portion of the input feature map (data of size 3x3x3). The size and number of weight kernels in each layer may vary depending on the application field and the number of feature map channels of input and output.

As shown, the weight kernel may be stored in a weight kernel cache 1910 that is separate from the processing unit 1930 for performing convolution of the convolution layer. In this case, bit quantization is applied to the weight kernel to reduce the number of bits of the element values (w ₁ , w ₂ , ..., w ₉ ) of the weight kernel, and then the size of the cache is adjusted according to the number of bits of the weight kernel. You can decide. In addition, the bit size of the multiplier or adder arranged in the processing unit 1930 that receives the element values of the weight kernel and the element values of the input feature map stored in the memory and performs multiplication and/or addition operations is also determined according to the result of bit quantization. It can be designed according to the number of bits of the weight kernel element value.

According to one embodiment, the input feature map cache 1920 may receive and store a portion of the input data (a portion corresponding to the size of the weight kernel). The weight kernel traverses the input data, and the input feature map cache 1920 can sequentially receive and store a portion of the input data corresponding to the position of the weight kernel. Part of the input data (x ₁ , x ₂ , ..., x ₉ ) stored in the input feature map cache 1920 and some element values of the weight kernel stored in the weight kernel cache 1910 (w ₁ , w ₂ , . .., w ₉ ) are each input to the corresponding multiplier 1932 and multiple multiplication is performed. The results of multiple multiplication by the multiplier 1932 are added by the tree adder 1934 and input to the adder 1940. When the input data consists of multiple channels (for example, when the input data is an RGB color image), the adder 1940 calculates the sum of the value stored in the accumulator 1942 (the initial value is 0) and the specific input channel. can be added and stored again in the accumulator (1942). The summed value stored in the accumulator 1942 can be added back to the summed value of the adder 1940 for the next channel and input to the accumulator 1942. This summation process of the adder 1940 and the accumulator 1942 is performed for all channels of input data, and the total sum value can be input to the output activation map cache 1950. The convolution procedure described above can be repeated for a portion of the input data corresponding to the weight kernel and the traversal position on the input data of the weight kernel.

As described above, when the element values of the weight kernel are stored in the weight kernel cache 1910 arranged outside the processing unit 1930, the number of bits of the weight kernel element values can be reduced by bit quantization according to the present disclosure. This has the effect of reducing the size of the weight kernel cache 1910 and the size of the multiplier and adder of the processing unit 1930. Additionally, as the size of the processing unit 1930 decreases, the power consumption of the processing unit 1930 may also be reduced.

FIG. 20 is a diagram illustrating an example of hardware implementation of an artificial neural network according to another embodiment of the present disclosure.

The depicted artificial neural network represents an example of hardware implementation of the convolution processing device 2000 of the convolutional layer of the convolutional artificial neural network. Here, the convolution layer performs convolution by applying a weight kernel of size 3x3x3 to a portion of the input activation map (data of size 3x3x3).

As shown, the weight kernel may be stored in a weight kernel cache 2010 that is separate from the processing unit 2030 for executing convolution of the convolution layer. In this case, bit quantization is applied to the weight kernel to reduce the number of bits of the element values (w ₁ , w ₂ , ..., w ₉ ) of the weight kernel, and then the size of the cache is adjusted according to the number of bits of the weight kernel. You can decide. In addition, the bit size of the multiplier or adder arranged in the processing unit 2030 that receives the element values of the weight kernel and the element values of the input activation map (or feature map) stored in the memory and performs multiplication and/or addition operations is also bit size. It can be designed according to the number of bits of the weight kernel element value according to the quantization result.

According to one embodiment, the input activation map cache 2020 may receive and store a portion (a portion corresponding to the size of the weight kernel) of input data composed of multiple channels (e.g., three RGB channels). The weight kernel traverses the input data, and the input activation map cache 2020 can sequentially receive and store a portion of the input data corresponding to the position of the weight kernel. A portion of the input data (x ₁ , x ₂ , ..., x ₂₇ ) stored in the input activation map cache (2020) and the element values of the weight kernel (w ₁ , w ₂ , . .., w ₂₇ ) are each input to the corresponding multiplier and multiple multiplication is performed. At this time, the kernel element values (w ₁ , w ₂ , ..., w ₉ ) of the weight kernel cache 2010 and the portion of the first channel of the input data stored in the input activation map weight cache 2020 (x ₁ , x ₂ , ..., x ₉ ) are input to the first convolution processing unit 2032. In addition, the weight kernel element values (w ₁₀ , w ₁₁ , ..., w ₁₈ ) of the weight kernel cache (2010) and the portion of the second channel of the input data stored in the input activation map cache (2020) (x ₁₀ , x ₁₁ , ..., x ₁₈ ) are input to the second convolution processing unit 2034. In addition, the weight kernel element values (w ₁₉ , w ₂₀ , ..., w ₂₇ ) of the weight kernel cache (2010) and the third channel portion (x ₁₉ , x ₂₀ , ..., x ₂₇ ) are input to the third convolution processing unit 2036.

Each of the first convolution processing unit 2032, the second convolution processing unit 2034, and the third convolution processing unit 2036 may operate identically to the processing unit 1930 shown in FIG. 19. The convolution results calculated by each of the first convolution processing unit 2032, the second convolution processing unit 2034, and the third convolution processing unit 2036 are summed by the tree adder 2038 and output. It may be input into the activation map cache 2040.

As described above, when the element values of the weight kernel are stored in the weight kernel cache 2010 arranged outside the processing unit 2030, the number of bits of the weight kernel element values can be reduced by bit quantization according to the present disclosure. This has the effect of reducing the size of the weight kernel cache 2010 and the size of the multiplier and adder of the processing unit 2030. Additionally, as the size of the processing unit 2030 decreases, the power consumption of the processing unit 2030 may also be reduced.

FIG. 21 is a diagram illustrating an example of hardware implementation of an artificial neural network according to another embodiment of the present disclosure.

The depicted artificial neural network represents an example of hardware implementation of the convolution processing device 2200 of the convolutional layer of the convolutional artificial neural network. Here, the convolution layer performs convolution by applying a weight kernel of size 3x3x3 to a portion of the input activation map (data of size 3x3x3).

As shown, the weight kernel may be implemented in a hardwired form within the processing unit 2220 for executing convolution of the convolution layer. In this case, after applying bit quantization to the weight kernel to reduce the number of bits of the element values (w _{1_K} , w _{2_K} , ..., w _{27_K} ) of the weight kernel, the size of the cache is adjusted according to the number of bits of the weight kernel. can be decided. In addition, the element values of the weight kernel implemented as a wire in the processing unit 2220 and the element values of the input activation map (or feature map) are input and are arranged in the processing unit 2030 to perform multiplication and/or addition operations. The bit size of the multiplier or adder can also be designed according to the number of bits of the weight kernel element value according to the result of bit quantization.

According to one embodiment, the input activation map cache 2210 may receive and store a portion (a portion corresponding to the size of the weight kernel) of input data composed of multiple channels (e.g., three RGB channels). The weight kernel traverses the input data, and the input activation map cache 2210 can sequentially receive and store a portion of the input data corresponding to the position of the weight kernel. Part of the input data (x ₁ , x ₂ , ..., x ₂₇ ) stored in the input activation map cache 2210 and element values (w _{1_K} , w _{2_K} ) of the weight kernel implemented as a wire in the processing unit 2220 , ... w _{27_K} ) are each input to the corresponding multiplier and multiple multiplication is performed. At this time, the weight kernel element values (w _{1_K} , w _{2_K} , ..., w _{9_K} ) implemented as wires in the processing unit 2220 and the first channel portion of the input data stored in the input activation map cache 2210 (x ₁ , x ₂ , ..., x ₉ ) is input to the first convolution processing unit 2222. In addition, weight kernel element values (w _{10_K} , w _{11_K} , ..., w _{18_K} ) implemented as wires in the processing unit 2220 and a portion of the second channel of the input data stored in the input activation map cache 2210 ( x ₁₀ , x ₁₁ , ..., x ₁₈ ) are input to the second convolution processing unit 2224. In addition, the weight kernel element values (w _{19_K} , w _{20_K} , ..., w _{27_K} ) of the weight kernel cache 2210 and the third channel portion of the input data stored in the input activation map cache 2210 (x ₁₉ , x ₂₀ , ..., x ₂₇ ) are input to the third convolution processing unit 2226.

The convolution results calculated by each of the first convolution processing unit 2222, the second convolution processing unit 2224, and the third convolution processing unit 2226 are summed by the tree adder 2228 and output. It may be entered into the activation map cache 2230.

As described above, when the element values of the weight kernel are implemented in a hardwired form within the processing unit 2220, the number of bits of the weight kernel element values can be reduced by bit quantization according to the present disclosure, and thus the processing This has the effect of reducing the number of wires implemented inside the unit 2220 and the size of the multiplier and adder of the processing unit 2220. Additionally, as the size of the processing unit 2220 decreases, the power consumption of the processing unit 2220 may also decrease.

FIG. 22 is a diagram illustrating the configuration of a system that performs bit quantization for an artificial neural network according to an embodiment of the present disclosure.

As shown, system 2300 may include a parameter selection module 2310, a bit quantization module 2320, and an accuracy determination module 2330. The parameter selection module 2310 can analyze the input configuration information of the artificial neural network. The configuration information of the artificial neural network includes the number of layers included in the artificial neural network, the function and role of each layer, information about the input and output data of each layer, the type and number of multiplication and addition performed by each layer, and the number of multiplications and additions performed by each layer. Type of activation function to be executed, type and configuration of weight kernel input to each layer, size and number of weight kernels belonging to each layer, size of output feature map, initial value of weight kernel (e.g., weight set as a real number) kernel element values), etc. may be included, but are not limited thereto. The configuration information of the artificial neural network may include information on various components depending on the type of artificial neural network (eg, convolutional artificial neural network, recurrent artificial neural network, multilayer perceptron, etc.).

The parameter selection module 2310 may refer to the input artificial neural network configuration information and select at least one parameter or parameter group to be quantized in the corresponding artificial neural network. How to select one parameter (or data) or group of parameters in an artificial neural network may be determined according to the effect of the selected parameter on the overall performance or computational amount of the artificial neural network (or the amount of resources required when implementing hardware). Selection of parameters can be performed by selecting any one of one weight, one feature map and activation map, one weight kernel, all weights belonging to one layer, all feature maps or activation maps belonging to one layer.

In one embodiment, in the case of the convolutional artificial neural network (CNN) 400 described above with reference to FIGS. 4 to 10, the convolutional layer 420 and/or the fully connected layer 440 are the CNNs 400. Since it has a large impact on the overall performance or amount of computation, the weight kernel or feature map/activation map of at least one of these layers 420 and 440 may be selected as one parameter to be quantized.

In one embodiment, at least one of a plurality of layers included in the artificial neural network can be selected and the entire weight kernel within that layer or the entire activation map data of that layer can be set as one parameter group, and the selection method is: the selected layer may be determined depending on the impact on the overall performance or computational amount of the artificial neural network, but is not limited to this and may include one of various methods. For example, the selection of at least one layer among the plurality of layers included in the artificial neural network is (i) sequentially from the first layer where input data is received to subsequent layers according to the arrangement order of the plurality of layers constituting the artificial neural network. (ii) a method of sequentially selecting from the last layer where the final output data is generated to the previous layer according to the arrangement order of the plurality of layers constituting the artificial neural network, (iii) the plurality of layers constituting the artificial neural network It may be executed by selecting the layer with the highest computational amount among the layers, or (iv) selecting the layer with the lowest computational amount among the plurality of layers that make up the artificial neural network.

When the selection of the data object to be quantized by the artificial neural network is completed by the parameter selection module 2310, information on the selected data is input to the bit quantization module 2320. The bit quantization module 2320 may refer to the input information of the selected parameter and reduce the data representation size for the corresponding parameter in bit units. Resources required for calculating the selected parameter may include, but are not limited to, a memory for storing the selected parameter or a data path for transmitting the selected parameter.

In one embodiment, when the bit quantization module 2320 reduces the data size of the selected parameter in bits, the weight kernel quantization and/or activation map quantization described with reference to FIGS. 4 to 13 may be performed.

When the bit quantization module 2320 completes bit quantization for the selected parameter, it transmits the information of the bit quantized artificial neural network to the accuracy determination module 2330. The accuracy determination module 2330 may reflect information on the bit-quantized artificial neural network in the configuration information of the artificial neural network input to the system 2300. The bit quantization module 2320 may determine whether the accuracy of the artificial neural network is greater than or equal to a predetermined target value, based on configuration information of the artificial neural network that reflects the information of the bit quantized artificial neural network. For example, the accuracy determination module 2330 reduces the size representing the data of the selected parameter in the artificial neural network in bits, and then reduces the accuracy of the output result of the artificial neural network (for example, the inference result of the artificial neural network). If it is above the pre-determined target value, it can be predicted that the overall performance of the artificial neural network can be maintained even if additional bit quantization is performed.

Therefore, when the accuracy determination module 2330 determines that the accuracy of the artificial neural network is greater than the target value, it transmits a control signal to the parameter selection module 2310, so that the parameter selection module 2310 selects another one included in the artificial neural network. Select a parameter or parameter group. Here, the method of selecting one parameter in the artificial neural network is (i) a method of sequentially selecting the next parameter of the previously selected parameter according to the arrangement order of each parameter or parameter group constituting the artificial neural network ("forward bit quantization ", forward bit quantization), (ii) a method of selecting the previous parameter of the previously selected parameter in the reverse direction according to the arrangement order of the parameters or parameter groups constituting the artificial neural network ("backward bit quantization", backward bit quantization), ( iii) a method of selecting the parameter with the highest computational cost after the previously selected parameter according to the order of computational amount among the plurality of parameters constituting the artificial neural network (“high computational cost bit quantization”), or (iv) artificial Among the plurality of parameters that make up the neural network, depending on the order of computation amount, it may be executed by selecting the parameter with the smallest computation amount next to the previously selected parameter (“low computational cost bit quantization”).

On the other hand, if the accuracy determination module 2330 determines that the accuracy of the artificial neural network is not more than the target value, it may determine that the accuracy of the artificial neural network has been degraded due to the bit quantization performed for the currently selected parameter. Therefore, in this case, the number of bits determined by the bit quantization performed immediately before can be determined as the final number of bits. In one embodiment, the accuracy of an artificial neural network refers to the probability that the artificial neural network will present the correct answer to the problem in the inference stage after learning how to solve a given problem (for example, recognition of an object included in an image as input data). It can mean. Additionally, the target value used in the bit quantization method described above can be expressed as the minimum accuracy to be maintained after bit quantization of the artificial neural network. For example, assuming that the threshold is 90 percent, even after reducing the memory size for storing the parameters of the selected layer by bit by bit by bit quantization, if the accuracy of the artificial neural network is more than 90 percent, additional bit quantization is performed. It can be run. For example, after performing the first bit quantization, if the accuracy of the artificial neural network has been measured to be 94 percent, additional bit quantization can be performed. After execution of the second bit quantization, if the accuracy of the artificial neural network is measured to be 88 percent, the result of the currently performed bit quantization can be ignored, and the number of data representation bits determined by the first bit quantization can be confirmed as the final bit quantization result. there is.

In one embodiment, when selecting a parameter or parameter group to perform bit quantization based on the computational cost according to a computational cost bit quantization method, the computational cost of each parameter may be determined as follows. In other words, when the sum of n bits and m bits is performed in a specific operation of an artificial neural network, the calculation amount of the operation is calculated as (n+m)/2. Additionally, when multiplication of n bits and m bits is performed in a specific operation of an artificial neural network, the calculation amount of the operation can be calculated as n x m. Therefore, the amount of computation for a specific parameter of an artificial neural network can be the result of adding up the amount of computation of all additions and multiplications performed on that parameter.

The method of selecting a specific parameter or parameter group in this bit quantization is to select the weight data or feature map and activation map data belonging to each layer, or each weight kernel belonging to one layer, or each weight data within one weight kernel. These can be selected as individual parameter groups.

For reference, the components shown in FIG. 22 according to an embodiment of the present disclosure may be implemented as software or hardware components such as FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit).

However, 'components' is not limited to software or hardware, and each component may be configured to reside on an addressable storage medium or may be configured to run on one or more processors.

Thus, as an example, a component may include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, properties, procedures, and sub-processes. Includes routines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables.

Components and the functionality provided within them may be combined into a smaller number of components or further separated into additional components.

Embodiments of the present disclosure may also be implemented in the form of a recording medium containing instructions executable by a computer, such as program modules executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and non-volatile media, removable and non-removable media. Additionally, computer-readable media may include both computer storage media and communication media. Computer storage media includes both volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transmission mechanism, and includes any information delivery medium.

Although the present disclosure has been described in relation to some embodiments in the specification, it should be noted that various modifications and changes can be made without departing from the scope of the present disclosure as can be understood by those skilled in the art. something to do. Additionally, such modifications and changes should be considered to fall within the scope of the claims appended hereto.

100: Artificial neural network 110_1 to 110_N: Layer
200: Artificial neural network 210: Input data
220: input layer 230: hidden layer
240: Output layer 400: Artificial neural network
410: Input data 420: Convolutional layer (CONV)
430: Subsampling layer (SUBS) 440: Fully connected layer (FC)
450: output data 510: input data
520: weight kernel 530, 260: row
540, 270: Columns 550, 280: Depth (channels)
610: first weight kernel 620: first result value
630: first activation map 710: second weight kernel
720: second result value 730: second activation map
810: input data 812: multiple product
814: Weight Kernel 816: Sum
818: Output data 820: First circuit section
822: Second traversal section 824: First feature map value
826: Second feature map value 910: Input data
912: Multiple product 914: Weight kernel
916: Sum 918: Output data
1010: Input data 1012: Multiple product
1014: Weight Kernel 1016: Quantization
1018: Quantization weight kernel 1020: Sum
1022: Output data 1024: Quantization
1026: Quantization activation map 1028: Weight kernel quantization
1030: Activation map quantization

Claims (20)

Bit quantization is performed by sequentially selecting any one of the plurality of layers constituting the artificial neural network, and the bit quantization performed on the selected layer by comparing the accuracy and target value of the artificial neural network A memory configured to determine whether the accuracy of the neural network has deteriorated, determine the final number of bits according to the judgment result, and store the quantized artificial neural network; and
A processing unit comprising a plurality of multipliers or a plurality of adders configured to process the quantized artificial neural network,
The one layer is selected from among the plurality of layers based on one of the following: arrangement order, amount of computation, and whether or not to search for accuracy change points for each layer,
The memory is,
If the accuracy of the artificial neural network is greater than the target value, the size of the data expression for storing the parameters of the selected layer is reduced in 1-bit units, and then additional bit quantization is performed, and the accuracy of the artificial neural network is greater than the target value. If it is less than that, the result of the bit quantization is ignored and the minimum number of bits that satisfied the target value in the previous bit quantization is determined as the final number of bits for the parameter of the selected layer,
The target value is,
Artificial neural network hardware that represents the minimum accuracy that must be maintained after bit quantization of the artificial neural network. According to claim 1,
The quantized artificial neural network is an artificial neural network hardware in which at least one of feature map data and activation map data for each of the plurality of layers is sequentially quantized based on a computation amount or a memory amount. According to claim 1,
The processing unit is configured to process a quantized artificial neural network in at least one of a computation bit quantization method, a forward bit quantization method, and a reverse bit quantization method. According to claim 1,
The computational amount and memory amount of the quantized artificial neural network are relatively reduced compared to before quantization, and the number of bits of at least one of the feature map data and activation map data for each of the plurality of layers stored in the memory is reduced. hardware. According to claim 1,
Artificial neural network hardware, wherein the memory includes at least one of buffer memory, register memory, and cache memory. According to claim 1,
Artificial neural network hardware, wherein the size of the data bits of the data path through which data of a specific layer among the plurality of layers is transmitted is reduced in bit units. According to claim 1,
Wherein the quantized artificial neural network is bit quantized to reduce the storage size of the memory configured to store at least one of feature map data and activation map data for each of the plurality of layers. According to claim 1,
Artificial neural network hardware, wherein the memory further includes a feature map cache. According to claim 1,
The processing unit further includes a tree adder configured to sum result values of multiple multiplication by the plurality of multipliers. According to claim 1,
Artificial neural network hardware further comprising an adder connected to the processing unit and an accumulator connected to the adder. According to claim 1,
Artificial neural network hardware further comprising an output activation map cache configured to store convolution result values of the processing unit. According to claim 1,
Artificial neural network hardware, wherein the processing unit further includes a plurality of convolution processing units. According to claim 12,
The processing unit further includes a tree adder configured to sum convolution result values of each of the plurality of convolution processing units. Bit quantization is performed by sequentially selecting any one of the plurality of layers constituting the artificial neural network, and the accuracy and target value of the artificial neural network are compared and the bit quantization performed on the selected layer is performed on the artificial neural network. a memory configured to determine whether the accuracy of has decreased, determine the final number of bits according to the determination result, store the quantized artificial neural network, and store the quantized feature map; and
It includes a processing unit configured to receive a weight kernel and the quantized feature map and process convolution,
The one layer is selected from among the plurality of layers based on one of the following: arrangement order, amount of computation, and whether or not to search for accuracy change points for each layer,
The memory is,
If the accuracy of the artificial neural network is greater than the target value, the size of the data expression for storing the parameters of the selected layer is reduced in 1-bit units, and then additional bit quantization is performed, and the accuracy of the artificial neural network is greater than the target value. If it is less than that, the result of the bit quantization is ignored and the minimum number of bits that satisfied the target value in the previous bit quantization is determined as the final number of bits for the parameter of the selected layer,
The target value is,
A convolution processing device that represents the minimum accuracy to be maintained after bit quantization of the artificial neural network. According to claim 14,
The processing unit further includes a multiplier and an adder,
A convolution processing device in which the bit sizes of the multiplier and the adder are designed to match the number of bits of the quantized feature map. According to claim 14,
The quantized feature map is a convolution processing device in which the feature map is sequentially quantized for each of the plurality of layers based on the amount of computation or amount of memory. According to claim 14,
For each of the plurality of layers, the number of bits of the quantized feature map is set to the number of bits immediately before the accuracy change point of each of the plurality of layers. According to claim 14,
For each of the plurality of layers, the quantized feature map is bit quantized by searching for accuracy change points for each of the plurality of layers. According to claim 14,
A convolution processing device in which bit quantization is performed on the feature map according to the number of bits of the weight kernel. Bit quantization is performed by sequentially selecting any one of the plurality of layers constituting the artificial neural network, and the accuracy and target value of the artificial neural network are compared and the bit quantization performed on the selected layer is performed on the artificial neural network. A memory that determines whether the accuracy of has decreased, determines the final number of bits according to the judgment result, and stores the quantized artificial neural network;
a processing unit configured to receive a weight kernel and the quantized feature map and process convolution;
a weight kernel cache configured to store weight kernel data of the artificial neural network;
an input feature map cache configured to store feature map data of the quantized artificial neural network; and
An output activation map cache configured to store output activation map data of the quantized artificial neural network,
The one layer is selected from among the plurality of layers based on one of the following: arrangement order, amount of computation, and whether or not to search for accuracy change points for each layer,
The memory is,
If the accuracy of the artificial neural network is greater than the target value, the size of the data expression for storing the parameters of the selected layer is reduced in 1-bit units, and then additional bit quantization is performed, and the accuracy of the artificial neural network is greater than the target value. If it is less than that, the result of the bit quantization is ignored and the minimum number of bits that satisfied the target value in the previous bit quantization is determined as the final number of bits for the parameter of the selected layer,
The target value is,
An apparatus representing the minimum accuracy to be maintained after bit quantization of the artificial neural network.

Images