KR102153791B1 - Digital neural, artificial neuron for artificial neuron network and inference engine having the same - Google Patents

Abstract

A digital neuron for an artificial neural network, an artificial neuron, and an inference engine including the same according to an embodiment of the present invention perform a multiplication operation with an input value by receiving an integerized weight in a summation method of partial products obtained by bit shift operation. By doing so, it can be implemented with small digital hardware, power consumption can be reduced, and operation speed can be greatly improved.

Description

Digital neurons for artificial neural networks, artificial neurons, and inference engines including the same {DIGITAL NEURAL, ARTIFICIAL NEURON FOR ARTIFICIAL NEURON NETWORK AND INFERENCE ENGINE HAVING THE SAME}

The present invention relates to a digital neuron for an artificial neural network, an artificial neuron, and an inference engine including the same.In particular, by simplifying a number of multiplication and addition operations based on integers, it is possible to improve computation speed and reduce power consumption, and to reduce power consumption. It relates to a digital neuron, an artificial neuron, and an inference engine including the same for an artificial neural network that can be implemented with digital hardware.

Recently, research on deep learning using an artificial neural network configured to process various information in a manner similar to that of the brain by simulating the way the human brain recognizes patterns is actively progressing. As an example, deep learning is applied to object classification, object detection, speech recognition, natural language processing and fields, and the field of application continues to expand.

An artificial neural network using a deep learning technique generally includes a plurality of artificial neurons, and the plurality of artificial neurons is configured to perform a dot product operation of a matrix (or vector). In other words, a number of artificial neurons constitute the inference engine of the artificial neural network and function as an operator.

In addition, the artificial neural network must perform learning based on vast amounts of training data before it is used for inference, so that it can exhibit the required pattern recognition performance. Therefore, in general, learning is performed on a server or a device equipped with a high-performance general-purpose computational accelerator such as GPU (Graphics Processing Unit)/TPU (Tensor Processing Units), and inference in actual use is performed in an embedded system. come. In addition, the artificial neural network performs a large amount of addition and multiplication operations during the learning and inference process.

However, when an artificial neural network is used for inference rather than learning, a large difference occurs in processing speed depending on the hardware performance of the applied system. In particular, in the case of an embedded system, since there are various limitations such as size and power consumption, there is a limitation in that it is difficult to use a high-performance general-purpose computational accelerator for an artificial neural network. Therefore, in an embedded system, artificial neurons of an artificial neural network are generally implemented in the form of an inference engine, which is digital hardware separately designed for inference. Nevertheless, the problem of the size, power consumption, and computational speed of the embedded system acts as a limitation on the design of the inference engine, making it difficult to apply the artificial neural network to small devices, and limiting the field of application of the artificial neural network. have.

Accordingly, there is a need for an optimized artificial neuron for an inference engine that is implemented in a small area and consumes low power and can significantly improve the computational speed for a large amount of computation required in an artificial neural network.

Korean Patent Publication No. 10-2016-0143505 (published on December 14, 2016)

An object of the present invention is to provide a digital neuron, an artificial neuron, and an inference engine including the same, implemented as digital hardware for an artificial neural network that can be implemented in a small size, consumes low power, and improves computation speed.

Another object of the present invention is to receive an integerized weight and perform a multiplication operation with an integer input value with a limited number of bit transitions and addition operations, thereby improving the operation speed, consuming low power, and reducing the realization area. It is to provide digital neurons, artificial neurons, and inference engines including the same for neural networks.

Another object of the present invention is to provide a digital neuron, an artificial neuron, and an inference engine including the same, capable of increasing the computation speed by simultaneously calculating the multiplication of a plurality of input values and a plurality of weights in parallel.

Another object of the present invention is to provide a digital neuron, an artificial neuron, and an inference engine including the same for an artificial neural network that can be implemented with a small area by separately performing an operation on a sign bit when partial products are summed.

Another object of the present invention is to perform computational processing in parallel for weight filters less than a predetermined size including a plurality of digital neurons, and divide the weight of the weight filter for weight filters having a size exceeding a predetermined size. Thus, it is intended to provide a digital neuron, an artificial neuron, and an inference engine including the same, in which the size of the weight filter can be adjusted by allowing computational processing to be performed simultaneously.

Another object of the present invention is to provide an inference engine that enables a plurality of layers to share digital neurons in an artificial neural network composed of a plurality of layers, thereby implementing a plurality of layers with a minimum number of digital neurons.

A digital neuron according to an embodiment of the present invention for achieving the above object includes an input vector including a plurality of input values in an integer format having a predetermined number of bits and a plurality of weights in an integer format having a predetermined number of bits. In a digital neuron implemented by digital hardware that performs an inner product operation by receiving a weight vector, the digital neuron includes a neuron, and the neuron receives a corresponding weight from the weight vector, and the applied integer format The weight of is decomposed into the sum of a plurality of partial products (R·2 ⁿ ) expressed as the product of a coefficient (R) having at least one value of -1, 0, 1 and a multiplier of 2 (2 ⁿ ), A plurality of weight decomposition units outputting a control signal according to a coefficient (R) and an index (n) of the decomposed partial product; A plurality of partial product generators for receiving a corresponding input value from the input vector, and outputting a plurality of partial products by bit shifting the input value in a direction of an upper bit by an index (n) in response to the control signal; And a partial product adder for summing the plurality of partial products in parallel to output a channel operation value. Includes.

Each of the plurality of partial product generators may additionally perform a one's complement operation or a zero conversion operation for the plurality of partial products according to the coefficient R.

The digital neuron may include a negative partial product counter for obtaining a count value by counting the number of times a one's complement operation is performed by the plurality of partial product generators; Further, the partial product adder may calculate the channel operation value by summing the count values together with the plurality of partial products.

The partial product adder calculates the channel operation value by summing the count value with the value excluding the value of the sign bit from the plurality of partial products, and adding the two's complement of the count value to the sign bit to the summation result. I can.

Each of the plurality of partial product generators includes a plurality of partial product calculators that are driven according to the control signal to calculate and output each partial product, and each of the plurality of partial product calculators bit the input value by the exponent (n). A shifting bit shifter; A one's complement operator for outputting the partial product by performing a one's complement operation on the output value of the bit shifter when the coefficient R is -1; And a zero multiplier for converting the partial product to 0 in response to the control signal when the coefficient R is 0. It may further include.

The partial product adder is composed of a plurality of stages each including a plurality of full adders, and a first stage of the plurality of stages receives the plurality of partial products, and the value of the same bit of the partial products of different preset numbers They are grouped and added, the remaining stages are added by grouping the addition values of the previous stage, and the final stage can calculate the channel operation value by adding the count value to the lower bits of the addition value of the previous stage.

The partial product adder receives the count value and performs a two's complement operation to calculate a count complement value; Further, the final stage may extend the sign bit of the channel operation value by adding the count complement value to the upper bit of the addition value of the previous stage.

The digital neuron includes a two-dimensional weight filter having a plurality of weights and one nerve element receiving a two-dimensional input feature map having a plurality of input values, and the partial product adder receives a predetermined bias value and calculates the channel. In addition to the value, an output value of the digital neuron may be output.

The digital neuron is a corresponding 2D weight filter among a plurality of 2D weight filters having a plurality of weights of the 3D weight filter and a corresponding 2D input feature map having a plurality of input values of the 3D input feature map. A channel adder that includes a plurality of neurons each receiving a dimensional input feature map, and outputs an output value of the digital neuron by adding the channel operation value output from each of the plurality of neurons and a predetermined bias value; It may further include.

In order to reduce the number of partial products, the weight may be applied by being substituted with a designated adjacent integer so that a predetermined integer is excluded.

A digital neuron according to an embodiment of the present invention for achieving the above object includes an input vector including a plurality of input values in an integer format having a predetermined number of bits and a plurality of weights in an integer format having a predetermined number of bits. In a digital neuron including a neuron implemented by digital hardware that performs a dot product operation by receiving a weight vector to be applied, the neuron is applied with a corresponding weight from the weight vector, and the weight of the applied integer format is- A plurality of weight decomposition units that decompose into a sum of a plurality of partial products (R·2 ⁿ ) expressed as a product of a coefficient (R) having at least one value of 1, 0, and 1 and a multiplier of 2 (2 ⁿ ); In the input vector, a corresponding input value, a negative input value that is two's complement of the input value, and a voltage corresponding to a 0 value are applied to select a value corresponding to the coefficient R, and the selected value is an index (n) A plurality of partial product generators for bit-shifting in the direction of a higher bit and outputting a plurality of partial products, respectively; And a partial product adder for summing the plurality of partial products in parallel to output a channel operation value. Includes.

Each of the plurality of partial product generators includes a plurality of partial product calculators for calculating and outputting each partial product, and each of the plurality of partial product calculators includes a negative input value and a zero value that are two's complement of the input value and the input value. A mux for receiving a voltage corresponding to a and selecting and outputting a value corresponding to the coefficient R; And a bit shifter for bit shifting a value selected and applied from the mux by the index (n). It may include.

Therefore, the digital neurons, artificial neurons, and the inference engine including the same for the artificial neural network of the present invention can be implemented with small digital hardware by allowing the digital neurons to perform multiplication operations in a bit shift and summation method, and reduce power consumption. Can be reduced, and the calculation speed can be greatly improved. In this case, the digital neuron provides a negative partial product using a two's complement operation to minimize the number of partial products, thereby further improving the operation speed and further reducing the size and power consumption.

In addition, it is possible to further improve the operation speed by performing a multiplication operation on multiple weights and multiple input values simultaneously in parallel.

In addition, the digital neuron can further simplify the circuit configuration by collectively adding a 2's complement operation for obtaining a plurality of negative partial products according to the number of negative partial products after a 1's complement operation. In addition, the digital neuron allows the operation of the sign bit to be separately performed when the partial products are summed, thereby minimizing the size and power consumption, and greatly increasing the operation speed on the screen.

In addition, the inference engine performs computational processing in parallel for weight filters less than a predetermined size including a plurality of digital neurons, and divides the weight of the weight filter for weight filters having a size exceeding the predetermined size. Can be performed, so that weight filters of various sizes can be used without changing the hardware structure.

In addition, the inference engine of an artificial neural network composed of multiple layers allows multiple layers to share digital neurons, thereby implementing multiple layers with the minimum number of hardware layers, minimizing the number of digital neurons included in the artificial neural network. In addition, a complex artificial neural network can be easily implemented with a minimum circuit configuration.

1 shows an example of a deep neural network among artificial neural networks.
2 shows an example of a mathematical model of an artificial neuron of an artificial neural network.
3 is a block diagram of a digital neuron of an artificial neural network according to an embodiment of the present invention.
4 is a diagram for explaining an error in the case of expressing a weight as a partial product of a specified number.
5 shows an example of a detailed configuration of a neuron in the artificial neuron of FIG. 3.
6 shows an example of a detailed configuration of the partial product adder of FIG. 5.
FIG. 7 is a diagram for explaining the operation concept of the two's complement operator of FIG. 6;
8 shows another example of a neuron of a digital neuron.
9 is a block diagram of digital neurons of an artificial neural network according to another embodiment of the present invention.
10 shows an example of a detailed configuration of a neuron in the artificial neuron of FIG. 9.
Fig. 11 shows an example of the detailed configuration of the partial product adder of Fig. 10;
12 shows the result of simulating power consumption of digital neurons according to the present embodiment.
13 shows an example of an artificial neuron structure capable of varying the size of a weight filter by using a digital neuron according to an embodiment of the present invention.
14 shows a schematic configuration of an inference engine of an artificial neural network according to another embodiment of the present invention.

In order to fully understand the present invention, the operational advantages of the present invention, and the objects achieved by the implementation of the present invention, reference should be made to the accompanying drawings illustrating preferred embodiments of the present invention and the contents described in the accompanying drawings.

Hereinafter, the present invention will be described in detail by describing a preferred embodiment of the present invention with reference to the accompanying drawings. However, the present invention may be implemented in various different forms, and is not limited to the described embodiments. In addition, in order to clearly describe the present invention, parts irrelevant to the description are omitted, and the same reference numerals in the drawings indicate the same members.

Throughout the specification, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components unless specifically stated to the contrary. In addition, terms such as "... unit", "... group", "module", and "block" described in the specification mean units that process at least one function or operation, which is hardware, software, or hardware. And software.

1 shows an example of a convolutional neural network among artificial neural networks, and FIG. 2 shows an example of a mathematical model of an artificial neuron of the artificial neural network.

1, in general, an artificial neural network, particularly a deep neural network (DNN), includes a plurality of layers, and each of at least one artificial neuron included in each of the plurality of layers is A predetermined operation is performed by receiving the output of the input layer or the previous layer as an input vector (X).

Each of the plurality of artificial neurons included in the plurality of layers may be configured to calculate and output the applied input vector (X) in a predetermined manner using a weight vector (w) designated in advance.

Referring to FIG. 2, elements (X ₁ , X ₂ , …, X _N ) of an input vector X are applied as input values to each of a plurality of artificial neurons. And the weight (w ₁ , w ₂ , …, w _N ), which is an element of the weight vector (w), is the corresponding input value among multiple input values (X ₁ , X ₂ , …, X _N ) of the input vector (X). It is a value to control the importance of

Here, a plurality of weights (w ₁ , w ₂ , …, w _N ) of the weight vector w may be designated to have various values by being varied by error backpropagation during the learning process of the deep neural network. The weight vector w designated by learning may be previously stored in each layer or in a separate memory.

For example, an artificial neuron has multiple input values (X ₁ , X ₂ , …, X _N ) of the input vector (X) and multiple weights of the weight vector (w), respectively (w ₁ , w ₂ , …, w _N ), the corresponding weight is multiplied, and the result of the multiplication is summed together with the bias value (b).

In other words, the artificial neuron can perform an operation that outputs an inner product of a weight vector (w) determined for the input vector (X), and the process of the mathematical model of this artificial neuron is expressed as Equation 1 I can.

Here, X _i , w _i , b and Output represent input values, weights, bias values, and output values, respectively.

In Equation 1, it is assumed that the input vector (X) and the weight vector (w) are each one-dimensional vector, but the input vector (X) and the weight vector (w) are multidimensional vectors (n-dimensional vectors (where n is a natural number)) May be, and may be expressed as a matrix. In this case, the weight vector w may have a dimension corresponding to the dimension of the input vector X. For example, when the input vector X is a two-dimensional matrix, the weight vector w may also be configured as a two-dimensional matrix.

FIG. 1 shows a convolutional neural network (CNN) as an example of an artificial neural network, and the convolutional neural network (CNN) is a neural network mainly used for image recognition, speech recognition, natural language processing, and handwriting recognition. Fig. 1 shows the schematic structure of LeNet-5, which is well known as a representative convolutional neural network.

As shown in FIG. 1, the convolutional neural network may include a feature extraction unit (Feature Extraction) and a classification unit (Classification) for extracting features of an input image.

In addition, the feature extraction unit of the convolutional neural network of FIG. 1 includes at least one convolution layer (C1, C2, C3) and at least one pooling layer (S1, S2, S3), and classification The unit may include at least one fully connected layer (FC1, FC2).

In FIG. 1, as an example, it is shown that the convolutional neural network includes three convolutional layers (C1, C2, C3), three pooling layers (S1, S2, S3), and two fully connected layers (FC1, FC2). , The number of layers can be variously adjusted.

The convolution layer (C1, C2, C3) receives at least one output feature map (or input image) output from the previous layer as an input vector (X) and performs a predetermined weight vector (w) and a convolution operation. To generate an output feature map.

In a convolutional neural network (CNN), the input vector X is called an input feature map, and the weight vector w is called a weight filter. That is, the convolutional layer of the convolutional neural network (CNN) receives an input feature map, filters it with a weight filter, and outputs an output feature map.

In addition, the pooling layers (S1, S2, S3) perform sub-sampling on the output values output from the previous layers (C1, C2, C3) in order to reduce overfitting of the artificial neural network and improve performance.

The fully connected layers FC1 and FC2 classify each of the feature maps extracted by the feature extraction unit into a known class.

The convolution layer (C1, C2, C3) receives a corresponding at least one input feature map from among at least one input feature map and includes a plurality of digital neurons that perform convolution operations using different pre-stored weight filters. I can. In this case, each of the plurality of convolutional layers may be configured to include a different number of artificial neurons.

The artificial neurons of the convolutional layers C1, C2, and C3 convolve the applied at least one input feature map with a predetermined weight filter. The artificial neurons of the convolutional layers (C1, C2, C3) move the weight filter on the input feature map while performing the convolution operation, and multiply the weight by the input value of the area where the weight filter is projected in the input feature map. Then, the multiplied result and the bias value (Bias) are summed, filtered with an activation function, and output. In this case, the output value is designated as the value of the designated location of the output feature map corresponding to the location where the weight filter is projected onto the input feature map. That is, the output feature map may be obtained as a vector expressed in a matrix form.

Therefore, the artificial neurons of the convolution layer (C1, C2, C3) can perform the convolution operation by dot product while moving the weight filter (w) specified for the input feature map (X), and the artificial neurons of the convolution layer. The process of can be expressed as in Equation 2.

Here, X, w, b, F, and Output represent input values, weights, bias values, and output values, respectively. Further, R and S denote the distances that the weight vector w can move in the row and column directions on the input vector X, respectively, and x and y denote the positions of the weight vector w with respect to the input vector X.

Referring to Equation 2, it can be seen that the artificial neuron of the convolutional neural network (CNN) also performs a dot product operation on the input vector (X) and the weight vector (w) as in Equation 1.

However, due to the nature of the convolution operation, the artificial neurons of the convolution layer (C1, C2, C3) repeat the multiplication of the input feature map a number of times corresponding to the size of the weight filter, and accumulate and add the results of the repeated multiplication. .

In general, in circuit implementation, the multiplication operation requires a larger power and operation time than the addition operation, and in particular, a larger area is required for the chip implementation. Therefore, digital neurons implemented as digital circuits as computation accelerators for artificial neural networks applied to embedded systems should be configured to efficiently perform multiplication operations.

3 is a block diagram of a digital neuron of an artificial neural network according to an embodiment of the present invention.

Referring to FIG. 3, the digital neuron according to the present embodiment is configured to include at least one neural element (NE).

The neural element NE receives the input feature map and the weight filter, and performs a predetermined operation on the applied input feature map and the weight filter and outputs it. Here, the predetermined operation may be, for example, an inner product operation or a convolution operation performed by accumulating inner product operations.

As described above, the convolution operation is the cumulative sum of the dot product of the input feature map and the weight filter, and the dot product is a number of input values (X _i ) that are elements of the input feature map and a number of weights (w _i = It is calculated as the sum of the multiplication values of w ₁ , w ₂ , …, w _N ).

In this case, a plurality of input values X _i may be applied in an integer format, and a weight w _i of the weight filter may also be applied as a weight in an integer format having a predetermined number of bits. As an example, the weight w _i may be a weight of a 5-bit integer format.

In general, artificial neural networks are weighted in 32-bit single-precision floating point format (FP32) or 16-bit half-precision floating point format (FP16) through the learning process. Is obtained. However, the structure of a multiplication circuit for multiplying a floating point is very complex, consumes large power, and requires a large area.

On the other hand, when the weight w _i is an integer, multiplication of the input value of the integer format and the weight may be performed by a combination of a bit shift operation and an addition operation. The bit shifter performing the bit shift operation and the adder performing the addition operation can be implemented in a very simple structure compared to the multiplication circuit, and thus can be implemented in a small size and greatly reduce power consumption. In particular, the operation speed is dramatically increased compared to the multiplication circuit.

However, even if the digital neuron uses the weight of the integer format rather than the weight of the floating point format, it should be considered whether the artificial neural network can exhibit the same performance. Accordingly, in this embodiment, the performance of the artificial neural network is simulated in the case of using the weight of the floating point format and the weight of the integer number. The simulation was performed using the Modified National Institute of Standards and Technology database (MNIST) in LeNet-5, a representative convolutional neural network developed for recognizing handwritten digits, and the simulation result is instead of the weight of the 32-bit single-precision floating-point format. It was confirmed that even if the weight of the 8-bit integer format is used, the accuracy of ReNet-5's handwritten number recognition is the same. In other words, it was confirmed that the same accuracy was obtained by using the weight of the 8-bit integer format, which has a smaller number of bits than the weight of the 32-bit floating-point format, and the performance of the artificial neural network remains the same.

Accordingly, in the present embodiment, the digital neuron receives an integerized weight so that the multiplication operation can be performed by a combination of a bit shift operation and an addition operation.

In particular, in the present embodiment, the digital neuron can receive the weight (w _i ) of the integer format with fewer bits (e.g., 5 bits) than the 8-bit integer format, which exhibits the same accuracy as the 32-bit single-precision floating-point format. have.

Artificial neural networks are mainly used for pattern recognition and classification, and these artificial neural networks do not require highly accurate arithmetic calculations. Rather, in artificial neural networks, a pooling layer is frequently added so that a variety of input data can be flexibly processed by preventing problems such as overfitting that are excessively optimized for training data.

Accordingly, in this embodiment, the number of bits is smaller than the 8-bit integer format having an accuracy corresponding to the weight of the 32-bit single-precision floating-point format so that the circuit of the neural element (NE) can be more simply implemented and power consumption is reduced. Here, as an example, a simulation was additionally performed for the case of receiving an integer weight of 5 bits). As a result of the simulation, it was confirmed that the performance degradation of the artificial neural network was insignificant even if the digital neuron was applied with a weight in the 5-bit integer format. Therefore, in the present embodiment, it is assumed that a digital neuron is applied with a weight w _i of an integer format having a number of bits less than an 8-bit integer that exhibits the same performance as a 32-bit single-precision floating-point format.

Here, the weight (w _i ) of the integer format having a small number of bits is obtained by multiplying each weight of the 32-bit single-precision floating-point format by a learning device of an artificial neural network by a predetermined integer value (eg, 16 (binary 10000)). , Can be obtained by discarding or rounding off the number of decimal places.

In addition, the weight w _i may be applied as a signed integer rather than a positive integer. That is, the weight w _i of the 5-bit signed integer format may be applied as a value ranging from -16 (11111) to 15 (01111).

In the present embodiment, since the weight (w _i ) has an integer value of a predetermined number of bits (here, as an example, 5 bits), the neural element (NE) performs a multiplication operation of the input value (X) and the weight (w _i ). It can be performed by a combination of a transition operation and an addition operation that adds a partial product calculated by a bit shift operation. Here, the bit shift operation is performed to calculate a partial product by bit shifting the input value X according to the value of each bit of the weight w _i .

Therefore, since the multiplication operation is performed by bit shifting and addition operation, the neural element NE improves the operation speed of digital neurons, reduces power consumption, and allows it to be implemented in a small size.

Referring to FIG. 3, the neuronal element NE includes N (where N is a natural number) partial product generator MBS and partial product adder PSUM.

The number (N) of partial product generators (MBS) is equal to the number of elements of the 2D weight filter, that is, the number of weights (w _i = w ₁ , w ₂ , …, w _N ). For example, when the weight filter is a matrix having a size of 5 X 5 (N = 25), the neural element NE includes 25 partial product generators MBS.

And the N partial product generators (MBS) are the partial products ((P1 ₁ , P2 ₁ ) of the corresponding input values (X) of the applied 2D input feature map and the corresponding weights (w _i ) of the 2D weight filter, (P1 ₂ , P2 ₂ ), …, (P1 _N , P2 _N )) are generated and output.

Partial product generators (MBS) is a weight (w) a group because of the integer value of the specified number of bits, the input value (X) and the weight (w _i) multiplying the weight (w _i) the input value (X) calculated in each Calculate the partial product ((P1 ₁ , P2 ₁ ), (P1 ₂ , P2 ₂ ), …, (P1 _N , P2 _N )) of two by bit shifting according to the bit value of. Here, multiple partial product generators (MBS) calculate partial products in parallel, so at the same time, multiple partial products ((P1 ₁ , P2 ₁ ), (P1 ₂ , P2 ₂ ), …, (P1 _N , P2 _N )) Can be calculated.

In this embodiment, each of the N partial product generators (MBS) generates and outputs two partial products ((P1 ₁ , P2 ₁ ), (P1 ₂ , P2 ₂ ), …, (P1 _N , P2 _N )). It is assumed that the partial product generator (MBS) generates two partial products will be described later.

Partial product adder (PSUM) is a partial product ((P1 ₁ , P2 ₁ ), (P1 ₂ , P2 ₂ ), …, (P1 _N , P2 _N )) that is transmitted by two from each of N partial product generators (MBS). The over-bias value BS is added, and an output value Out, which is a result of the dot product operation of the 2D input feature map and the 2D weight filter, is output. Here, the output value Out output from the partial product adder PSUM is the result of the dot product operation, not the result of the convolution operation, because the 2D weight filter is a result of the operation with a part of the 2D input feature map. In addition, the bias value BS denotes the bias value b of Equations 1 and 2.

In the case of a conventional GPU/TPU, an operation value is calculated by multiplying and accumulating one weight w _i and a corresponding input value X in a weight filter during one cycle. Therefore, iterative operation must be performed during the cycle corresponding to the number (N) of the weights (w _i ), which are elements of the weight filter. That is, for N cycles, iterative operations must be performed.

On the other hand, in the digital neuron according to the present embodiment, the N partial product generators MBS of the neural element NE perform calculations simultaneously in parallel, and thus, the calculation can be performed at high speed. In addition, since the partial product generator (MBS) calculates the partial product using a bit shift method, the digital circuit is simple to implement, thereby improving the operation speed, reducing power consumption, and implementing a small size.

Meanwhile, since the neural element NE according to the present embodiment is configured to receive a weight in the signed integer format, the partial product generator MBS can generate a negative partial product in which the sign of the partial product is negative. In the case of using the negative partial product, the number of partial products to be generated for the multiplication operation of the input value X and the weight w _i can be further reduced.

For example, if the weight (w _i ) is 7, the multiplication operation of the weight (w _i ) and the input value (X) is 7(00111)·X. And when the multiplication operation of the weight (w _i ) and the input value (X) is performed by the addition of the partial product and the partial product using the bit shift operation, the value of each bit of the weight (w _i ) (that is, a multiplier of 2) (2 ⁿ )) is the sum of the partial product of the input value (X) and is calculated as 4(00100)·X + 2(00010) ·X +1(00001)·X. That is, three partial products must be generated. Here, the partial product may be obtained by shifting the input value X by a bit having a value of 1 of the weight w _i (an exponent ⁿ of a multiplier of 2 (2 ⁿ )) in the upper bit direction. That is, it requires three bit shifting circuits and an adder.

However, when the neuron (NE) can generate a negative partial product, the multiplication of the weight (w _i ) 7 and the input value (X) can be calculated as 8(01000)·X + (-X). Here, the negative partial product -X may be obtained by performing a two's complement operation on the input value X. That is, it can be calculated using two partial products and two's complement. Therefore, the same operation can be performed with two bit shift circuits, a two's complement circuit, and an adder.

In this way, when the weight (w _i ) is reconstructed so that the number of partial products is minimized by using partial products and two's complement, the positive integer (1 ~ 15) of the 5-bit weight (w _i ) and The number of all cases for multiplication of the input value (X) is shown in Table 1.

In the weight (w _i ) of the signed integer format of Table 1, the binary number 10000 means -16, but when used to calculate the weights 14 and 15 as in Table 1, the binary number 10000 is considered as +16.

Referring to Table 1, the multiplication of the weight (w _i ) and the input value (X) within a range of 1 to 15 may be calculated as a sum of up to three partial products. In addition, when a negative partial product is required in Table 1, it can be obtained by performing a two's complement operation on the calculated partial product.

A typical multiplication operation between the weight (w _i ) having a 5-bit integer value and the input value (X) requires adding up to 5 partial products. On the other hand, as shown in Table 1, when a negative partial product is used, it is only necessary to add up to three partial products, so that the configuration of a circuit for performing a multiplication operation can be simplified.

On the other hand, as shown in Table 1, only 11 and 13 are the weights that require the maximum number of 3 partial products in multiplication of the weight (w _i ) having a 5-bit integer value and the input value (X), and the remaining weights (1 ~ 10 , 12, 14, 15) and the input value (X) can be calculated as the sum of two or less partial products.

Accordingly, the multiplication of the input value X and the remaining weights excluding the weights 11 and 13 that require the maximum number of partial products among the weights w _i having a 5-bit integer value may be expressed as Equation 3.

Referring to Equation 3, a weight (w _i ) having a 5-bit integer value is a coefficient (w _b1 , w _b2 ) having a value of -1, 0 or 1 and a multiplier of 2 (2 ⁱ , 2 ). ^j ) is expressed as the sum of the partial products (w _b1 2 ⁱ , w _b2 2 ^j ), and the value of -1 among the coefficients (w _b1 , w _b2 ) can be expressed as 2's complement (-1) to 1 have.

Therefore, the multiplication of the weight (w _i ) and the input value (X) is equal to two partial products (w _b1 2 ⁱ ·X, w _b2 2 ^j ·X) and two's complement (bit values (w _b1 , w _b2 )). 1) indicates that it can be calculated as the sum of the combinations.

On the other hand, if the weight (w _i ) is in the range of -1 to -15, the product of each input value (X) of the weight (w _i ) within the range of 1 to 15 is calculated as shown in Table 2. I can. Therefore, excluding the weights -11 and -13, the negative weight (w _i ) can also be calculated by summing the combination of the two partial products and the two's complement of the power of 2 (2 ⁿ ).

In Table 2, it can be seen that only -11 and -13 need three partial products.

Accordingly, in order to simplify the structure of the neural device (NE), the digital neuron according to the present embodiment has a weight (w) of the integer format of the specified number of bits excluding at least one predetermined integer (in this case, ±11 and ±13). _i ) can be configured to be authorized.

As a result, the multiplication of the weight (w _i ) and the input value (X) of the 5-bit range can be performed even if only two bit shift circuits and two's complement circuits are provided to perform two partial products for the multiplier of 2 (2 ⁿ ). Since it can be performed, the structure of the neural element (NE) can be simplified as much as possible. That is, each of the N partial product generators MBS may be configured to output two partial products ((P1 ₁ , P2 ₁ ), (P1 ₂ , P2 ₂ ), …, (P1 _N , P2 _N )).

Even if the weight (w _i ) is designated as a value in the range of 6 bits or more, similarly, the weight (w) and the input value (X) are equally determined using a two or three bit transition circuit and a two's complement circuit. You can make it do multiplication.

And separately from this, when the weight (w _i ) is 0, a circuit that converts to 0 regardless of the input value (X) may be included in the nerve element (NE). For example, when the input value (X) is applied, it is It may include a logic circuit that performs an AND operation.

Although the 5-bit weighting (w _i) of -16 to 15 degrees is excluded ± 11 and ± 13, but this value may be selected as a weight (w _i) limited, this limitation of the weight (w _i) ANN The effect on the performance of is not significant.

4 is a diagram for explaining an error in the case of expressing a weight as a partial product of a specified number.

In FIG. 4, for example, for a case in which the weight w is an 8-bit integer format, only a range of positive integers from 0 to 128 is considered, and a case where all the weights w _i within the range are expressed and a multiplier of 2 (2 Excluding some integers to be expressed as a combination of two partial products of ⁿ ) and two's complement, and excluding some integers to be expressed as a combination of three partial products of 2's (2 ⁿ ) and two's complement Represents the error of

In Fig. 4, the X-axis represents the actual weight (w _i ) of the 7-bit positive integer format, and the Y-axis represents some values so that the weight (w _i ) of the 7-bit positive integer format is expressed as the sum of two or three partial products. Represents the weights that are excluded and replaced with adjacent integer values. Here, green indicates a weight expressed as the sum of three partial products, and blue indicates a weight expressed as the sum of two partial products.

However, as shown in Fig. 4, although some values are excluded and replaced with adjacent integer values so that the weight (w _i ) of the 7-bit positive integer format is expressed as the sum of two or three partial products, two partial products When expressed as the sum of, the maximum error is approximately 9.4%, and when expressed as three partial products, it is approximately 2.3%. That is, even if the weight (w _i ) of the 7-bit positive integer format is not expressed as an integer in the range of 0 to 128, it is expressed by replacing it with an integer that can be expressed as the sum of two partial products or the sum of three partial products. Is not big.

In FIG. 4, only the positive integer range (0 to 128) is considered, but it is obvious that the negative integer range (0 to -128) can be expressed in the same manner. That is, when the weight is an 8-bit coded integer, the error appears to be less than 10% even if it is replaced with an integer that can be represented by two partial products or three partial products.

Therefore, through simulation in advance, the error of the artificial neural network according to the change in the number of partial products representing the weight (w _i ) is reviewed, and if the reviewed error is within a predetermined tolerance, the number of digital neurons is small. It may be configured to receive a weight (w _i ) expressed as an integer expressible by a combination of a partial product of and two's complement.

Table 3 shows the results of simulating the guessing accuracy of LeNet-5, one of the deep convolutional neural networks, discriminating the written number provided by MNIST according to the weight (w _i ).

Referring to Table 3, it can be seen that when the weight w _i is in the 32-bit single-precision floating-point format and the 8-bit integer format, the guessing accuracy of LeNet-5 is the same as 99.10%. In addition, when the weight w _i is in a 5-bit integer format, the accuracy is 98.95%, and in the case of the 5-bit integer format excluding ±11 and ±13, the accuracy is 98.92%. That is, even if the weight w _i is applied in a 5-bit integer format excluding ±11 and ±13, it can be seen that the effect on the performance of the artificial neural network is not significant.

The artificial neural network learning device converts the weight of the floating-point format into a 5-bit integer and transfers it to the digital neuron of the artificial neural network. When the weight is acquired with a specified value (±11, ±13) to be excluded from the conversion process, this It can be converted to another adjacent integer and passed. For example, when the learned weights w _i are 10.7 and 11.3, the learning device may convert them into 10 and 12, respectively.

Accordingly, the weight w _i applied to the neural element NE of the digital neuron may be an integer having a predetermined number of bits excluding some specified number as described above.

Since the weight (w _i ) can be expressed as the sum of two partial products, each partial product generator (MBS) will be implemented by including two partial product circuits, a two's complement circuit, and a weight 0 calculator as shown in Equation 3. I can. And each of the N partial product generators (MBS) uses two partial products ((P1 ₁ , P2 ₁ ), (P1 ₂ , P2 ₂ ), …, (P1 _N , P2 _N )) as a partial product adder (PSUM). Print.

Partial product adder (PSUM) is 2N ((P1 ₁ , P2 ₁ ), (P1 ₂ , P2 ₂ ), …, (P1 _N , P2 _N )) and bias values applied from N partial product generators (MBS). The output value (Out) is output by summing (BS).

Here, the output value Out is a result of a dot product operation on a partial area of the 2D input feature map by the 2D weight filter, and is an element of the 2D output feature map.

In order to perform a convolution operation, the artificial neuron may perform a dot product calculation of a 2D input feature map corresponding to the weight filter by moving the 2D weight filter on the 2D input feature map.

5 shows an example of a detailed configuration of a neuron in the artificial neuron of FIG. 3.

Referring to FIG. 5, the neural element NE of a digital neuron is described, wherein the neural element NE includes N weight decomposition units (WDC), N partial product generators (MBS), partial product adder (PSUM), and negative. It includes a partial product counter (NCCP).

As described above, the neural element NE receives the 2D input feature map and performs the dot product operation using the 2D weight filter. In addition, N input values (X) of the 2D input feature map have m-bit (e.g., 8-bit) integer values, and N weights (w _i ) of the weight filter are specified integers (e.g., ± It is assumed that 11 and ±13) have an integer value of the specified number of bits n (eg, 5 bits) excluding.

Each of the N weight decomposition units (WDC) receives the corresponding weight (w _i ) among the N weights, and the partial product (w _b1 ) of the coefficients (w _b1 , w _b2 ) and the multiplier of 2 (2 ⁱ , 2 ^j ) 2 ⁱ , w _b2 2 ^j ), and the operation of the corresponding partial product generator (MBS _i ) among N partial product generators (MBS) according to the decomposed weight (w _b1 2 ⁱ , w _b2 2 ^j ) Control. At this time, the weight decomposition unit (WDC) is a weight weight (w _i) depending on the power of 2 (2 ⁿ⁾ and a table consisting of a combination of the two's complement 1 and 2 so that the number of the minimum of the (w _i) the partial products Decomposition and determine the operation of the partial product generator (MBS _i ). And by controlling the partial product generator (MBS _i) to perform the determination operation, so that partial product generators (MBS _i) is able to perform the multiplication of the weight (w _i) and the input value (X).

The weight decomposition unit (WDC) determines the number of bits to which the input signal (X) is to be shifted according to the weight (w _i ) using Tables 1 and 2. Then, the shift control signals SH1 and SH2 are output according to the determined number of bits. Here, the number of bits to be shifted corresponds to the exponent (i, j) in a power of 2 (2 ⁱ , 2 ^j ) of the decomposed weight (w _i ).

In addition, the weight decomposition unit WDC determines a value for which a negative partial product is required, that is, a two's complement, from the coefficients w _b1 and w _b2 of the weight w _i decomposed from Tables 1 and 2. When the coefficients w _b1 and w _b2 are negative, that is, -1, the weight decomposition unit WDC determines that it takes 2's complement. In addition, the inversion control signals INV1 and INV2 are output according to the required value for the negative partial product.

Meanwhile, the weight decomposition unit WDC outputs the zero operation signals ALZ1 and ALZ2 when the coefficients w _b1 and w _b2 of the decomposed weight w _i have a value of 0. Further, the weight decomposition unit WDC transmits all inversion control signals INV1 and INV2 applied to the N partial product generators MBS to the negative partial product counter NCCP.

The weight decomposition unit (WDC) generates shift control signals (SH1, SH2), inversion control signals (INV1, INV2), and zero operation signals (ALZ1, ALZ2) according to the weights (w _i ) based on Tables 1 and 2. It may include at least one look-up table for doing so. In FIG. 5, as an example, the weight decomposition unit WDC includes three lookup tables for generating shift control signals SH1 and SH2, inversion control signals INV1 and INV2, and zero operation signals ALZ1 and ALZ2, respectively. It is shown assuming that.

Each of the N partial product generators MBS may perform a multiplication operation of an input value X and a weight w _i by performing a partial product for each bit and a two's complement operation as shown in Equation 3. Accordingly, each of the N partial product generators MBS includes two partial product calculators PP1 and PP2. In this embodiment, as it is assumed that the weight decomposition unit (WDC) decomposes the weight (w _i ) into two partial products (w _b1 2 ⁱ , w _b2 2 ^j ), each of the N partial product generators (MBS) It is shown to include two partial product calculators (PP1, PP2). However, when the weight decomposition unit (WDC) decomposes the weight (w _i ) into three or more partial products (w _b1 2 ⁱ , w _b2 2 ^j ), each of the N partial product generators (MBS) It is configured to include a partial product calculator of a number corresponding to the number of partial products.

Two partial product calculators (PP1, PP2) include a bit shifter (BSH), one's complement operator (1CMP), and a zero multiplier (ZOP), respectively, to calculate a specific bit of the weight (w _i ) and an input value (X). Partial product and partial product (P1, P2) are output.

In response to the shift control signals SH1 and SH2 applied from the weight decomposition unit WDC, the bit shifter BSH makes the input value X bit shift in the upper bit direction. That is, the input value (X) is multiplied by the decomposed weight (w _b1 2 ⁱ , w _b2 2 ^j ) by a power of 2 (2 ⁱ , 2 ^j ), that is, a specific bit. Here, the bit shifter BSH may be implemented as a barrel shifter, for example. The bit shifter (BSH) corresponds to the exponent (i, j) of a multiplier of 2 (2 ⁱ , 2 ^j ) in the decomposed weight (w _b1 2 ⁱ , w _b2 2 ^j ) as shown in Tables 1 and 2. Thus, the input value (X) is bit shifted in the upper bit direction. The number of bits q of the shift control signals SH1 and SH2 is determined according to the maximum range that the bit shift BSH shifts with respect to the weight w _i shown in Tables 1 and 2.

In addition, the one's complement operator 1CMP calculates one's complement by inverting the values applied from the bit shifter BSH in response to the inversion control signals INV1 and INV2 applied from the weight decomposition unit WDC.

The zero multiplier ZOP converts and outputs a value applied from the one's complement operator 1CMP to 0 in response to the zero operation signals ALZ1 and ALZ2 applied from the weight decomposition unit WDC. The zero multiplier ZOP is a component added to output a value of 0 regardless of the input value X when the weight w _i is applied as 0.

Here, the reason that the two partial product calculators PP1 and PP2 include a one's complement operator instead of a two's complement operator is to further simplify the structure of the partial product calculators PP1 and PP2.

The two's complement operator must perform an operation that adds 1 after inverting all the applied bit values. That is, the two's complement operator requires an adder along with an inversion circuit that performs bitwise inversion of the applied value.

On the other hand, the 1's complement circuit simply inverts and outputs all the bit values of the applied value, so there is no need for an adder. Therefore, compared to the two's complement circuit, the one's complement circuit has a simpler circuit configuration.

The size or power consumption of one adder is not large. However, since the neuron (NE) contains N partial product generators (MBS), and each of the N partial product generators (MBS) contains two partial product calculators (PP1, PP2), artificial neurons are N * 2 each. It should include a two's complement circuit. Therefore, by converting the 2's complement circuit to the 1's complement circuit, it is possible to reduce N * 2 adders in one neuron (NE) of a digital neuron. That is, the size of artificial neurons can be greatly reduced, the computational speed can be improved, and power consumption can be reduced.

However, as the partial product calculators PP1 and PP2 include one's complement operator instead of the two's complement operator, the sum of the partial products P1 and P2 output from the partial product calculators PP1 and PP2 is the weight w _i ) And the multiplication value of the input value (X) and the difference may occur. This difference occurs because 1 is not added because 1's complement, not 2's complement, is used when calculating the negative partial product among the partial products P1 and P2.

Accordingly, in this embodiment, a negative partial product counter (NCCP) is separately included so that the partial product calculators PP1 and PP2 perform a one's complement operation instead of two's complement to calculate one that is not added. The negative partial product counter NCCP counts the inversion control signals INV1 and INV2 applied from the weight decomposition unit WDC. Here, the counted value means the number of negative partial products required according to Tables 1 and 2 from the N weights w _i . That is, the negative partial product counter NCCP collectively counts a value of 1 to be added to a two's complement operation in the entire neuro element NE to obtain a count value NUM_P. Then, the count value NUM_P is transferred to the partial product adder PSUM.

The partial product adder (PSUM) receives two partial products (P1 _i , P2 _i ) from each of the N partial product generators (MBS), and receives a count value (NUM_P) from the negative partial product counter (NCCP), The bias value BS is applied, all of them are summed, and the output value Out is output.

6 shows an example of a detailed configuration of the partial product adder of FIG. 5.

As described above, the partial product adder PSUM receives two partial products P1 _i and P2 _i from each of the N partial product generators MBS. Here, it is assumed that N is 25. If N is 25, the partial product adder (PSUM) calculates 50 (= 2N) partial products ((P1 ₁ to P1 _N ), (P2 ₁ to P2 _N )) and count values (NUM_P) and bias values (BS). It is applied and summed, and the output value (Out) is output.

In this embodiment, the partial product adder (PSUM) basically performs an addition operation in a manner similar to that of the Wallace Tree Adder. In this embodiment, the partial product adder (PSUM) includes a plurality of stages (ST1 to STk) and a CLA adder (CAL) corresponding to the number of partial products to be added, and each of the plurality of stages (ST1 to STk) is It includes a number of addition groups (GPs) including the full adder (FA) of.

In the first stage (ST1), each of the plurality of addition groups (GP) is added by grouping the bits at the same position by a preset number of 2N partial products ((P1 ₁ ~ P1 _N ), (P2 ₁ ~ P2 _N )). do. When 2N partial products ((P1 ₁ ~ P1 _N ), (P2 ₁ ~ P2 _N )) are j-bits (j is a natural number, for example, j = m+n), each addition group (GP) is It may include j full adders (FA).

For example, the full adders FA of the first addition group GP of the first stage ST1 may add the same bits of the three partial products P1 ₁ , P1 ₂ , and P1 ₃ as inputs. Further, the full adders FA of the second addition group GP may also add the same bits of the three partial products P1 ₄ , P1 ₅ , and P1 ₆ as inputs.

However, in some cases, the full adders FA of each of the plurality of addition groups GP of the first stage ST1 use the same bit of the _five partial products P1 ₁ , P1 ₂ , P1 ₃ , P1 ₄ , and P1 ₅ . It can also be added by being authorized as an input. That is, the number of additions by each addition group GP receiving the same bit of the partial product may be variously adjusted.

When the first stage ST1 performs addition by grouping 2N partial products ((P1 ₁ to P1 _N ), (P2 ₁ to P2 _N )) by a, A(= 2N) is performed in the first stage ST1 /a) addition groups (GPs) are included.

In addition, the stages after the second stage ST2 receive the results added from the previous stage, group a pieces again, and perform the addition. At this time, as shown in FIG. 4, since the full adder FA receives an input of a-bit and outputs 2 bits of the carry and sum value, each stage ST2 to STk-1 is an addition group ( The number of GP) decreases sequentially by 2/a.

For example, when the full adder (FA) of each addition group (GP) is configured to receive an input of 3 bits and output 2 bits, and the number of partial products (2N) is 50, the first stage (ST1) is 17 (50). /3 = 16.6) addition groups (GPs). In addition, the second stage ST2 includes 12 addition groups GP (17*2/3 = 11.3), and the third stage ST3 includes 8 addition groups GP. Since the number of addition groups GP is reduced by 2/3 for each stage, the number of addition groups GP becomes 2 in the seventh stage ST7.

And while the addition result is transferred to each stage (ST1 to STk), the bit width of the addition result is extended by 1 bit. As an example, when 2N partial products ((P1 ₁ to P1 _N ), (P2 ₁ to P2 _N )) are j-bit values, each of the A addition groups GP of the first stage ST1 is j Includes a full adder (FA) and outputs A j-bit addition values. However, each of the B addition groups GP of the second stage ST2 includes j+1 full adders FA and outputs B j+1-bit addition values. In addition, each of the C (C = 2B/a) addition groups GP of the third stage ST3 includes j+2 full adders FA and outputs C j+2 bit addition values.

The method of adding the 2N partial products ((P1 ₁ to P1 _N ), (P2 ₁ to P2 _N )) of the partial product adder PSUM is similar to the Wallace tree adder. However, the structure of this partial product adder (PSUM) is the same when the input 2N partial products ((P1 ₁ ~ P1 _N ), (P2 ₁ ~ P2 _N )) are natural numbers, and the sign bit for the negative partial product is considered. It is not done.

When the partial product adder (PSUM) calculates the sum of individual signed numbers and outputs the channel operation value (MO) including the sign bit (s), 2N partial products of j bits ((P1 ₁ to P1 _{In each of N} ) and (P2 ₁ ~ P2 _N )), the sign bit (1 bit) is extended by s bits (for example, 6 bits) and must be input as a value of j+s bits. That is, the s bit (s = T-1) must be additionally extended as a sign bit.

In this case, in each stage (ST1 to STk) of the partial product adder (PSUM), each addition group (GP) must further include s full adders (FA), which is the circuit area of the partial product adder (PSUM). There is a problem that increases by about 21%.

Accordingly, the partial product adder PSUM according to the present embodiment receives the value of the sign bit to be extended separately and adds it in the final stage STk, thereby reducing the size and power consumption of the partial product adder PSUM. .

To this end, the partial product adder (PSUM) further includes a two's complement operator (2CMP) for converting the count value (NUM_P) applied from the negative partial product counter (NCCP) into two's complement, and a two's complement operator (2CMP). ) Calculates two's complement to the count value NUM_P, and outputs the count complement value N_NUM_P. At this time, the number of bits T of the count complement value N_NUM_P has the number of bits corresponding to the number of extended sign bits.

The two's complement operator (2CMP) converts the count value (NUM_P) into two's complement, which converts the count value (NUM_P) into 2's complement of 2N partial products ((P1 ₁ to P1 _N ), (P2 ₁ to P2). _{This is because N} )) is equal to the sum of all the sign bits of the negative partial product.

FIG. 7 is a diagram for explaining the operation concept of the two's complement operator of FIG. 6;

In FIG. 7 as an example, it is assumed that the partial product adder (PSUM) adds and outputs six partial products (11101, 10110, 10010, 00101, 10101, 10000) in a 5-bit binary integer format. In FIG. 7, five partial products (11101, 10110, 10010, 10101, 10000) with a sine bit of 1 are negative and are negative partial products, and one partial product with a sine bit of 0 (00101) is positive. Is a positive partial product.

And, in the six partial products, the sign bit is further extended by s bits (for example, 6 bits) and converted into a value of j+s bits (11 bits in FIG. 7). The extended sign bit is eventually extended with the same bit value as the sign bit.

However, since the value of the entire extended sign bit is the sum of 5 values of -1 and 1 value of 0, it is equal to (-1) X 5 + 0 X 1 = -5. That is, it is equal to the number of negative numbers, that is, the number of negative partial products (5) and the value of 2's complement (-5).

When adding 6 partial products (11101, 10110, 10010, 00101, 10101, 10000) of the 5-bit binary integer format, 2 bits of the positive binary integer format excluding the sign bit part are added, and then, the negative part It means that it is the same even if the number of products (5) and the value (-5) obtained by taking the complement of 2 are separately added.

Accordingly, the partial product adder (PSUM) according to the embodiment of the present invention includes a two's complement operator (2CMP), and converts the count value (NUM_P) applied from the negative partial product counter (NCCP) into two's complement, and is extended. By acquiring the count complement value N_NUM_P having the number of bits corresponding to the number of coded bits, the size and power consumption of the partial product adder PSUM can be reduced.

Referring back to FIG. 6, the count complement value N_NUM_P output from the 2's complement operator 2CMP corresponds to the upper bit of the summation result accumulated from the last stage STk to the previous stage, as shown in FIG. It is added by being applied as one of a plurality of inputs of the full adder (FA).

Meanwhile, in the partial product adder PSUM according to the present embodiment, since the partial product calculators PP1 and PP2 of the partial product generator MSB perform partial multiplication using one's complement rather than two's complement, a negative partial product counter ( NCCP) must be added to the count value (NUM_P). Accordingly, as shown in FIG. 4, each bit of the count value NUM_P is applied as an input of a plurality of inputs of the full adder FA corresponding to the lower bit of the last stage STk and added.

In addition, the partial product adder PSUM receives the bias value BS applied to the stage before the final stage STk and adds the bias value BS. At this time, as shown in FIG. 6, the bias value BS may be converted into a value of a binary format having an additionally extended number of bits (BS'[j-1+s:0]) and applied as shown in FIG. . Each bit of the bias value BS' of the binary format having an additional extended number of bits is applied to one of a plurality of inputs of the full adder FA and added.

In addition, the CLA adder CLA receives and adds each bit value added in the final stage STk, and finally outputs an output value Out[j+s:0].

As a result, the partial product adder (PSUM) according to the present embodiment can perform high-speed calculation using a Wallace tree addition method capable of a high-speed addition operation. In addition, the operation on the sign bit is calculated as two's complement to the count value NUM_P applied from the negative partial product counter NCCP, and is added to the upper bit separately, thereby reducing circuit size and power consumption. In addition, by adding the count value (NUM_P) to the lower bit, the partial product calculators (PP1, PP2) of the partial product generator (MBS) can have a one's complement operator (1CMP) instead of a two's complement operator (2CMP). do.

8 shows another example of a neuron of a digital neuron.

The neural element NE of FIG. 8 is similar to the neural element NE of FIG. 5, with N weight decomposition units (WDC), N partial product generators (MBS), partial product adder (PSUM), and negative partial product counter (NCCP). ).

Each of the N weight decomposition units (WDC) receives a corresponding weight (w _i ) among N weights, and a coefficient (R _a , R _b , R _c ) and a multiplier of 2 (2 ^a , 2 ^b , 2 ^c ) Partial product of (R _a ·2 ^a , R _b ·2 ^b , R _c ·2 ^c ) decomposition into the form (w _i = R _a ·2 ^a + R _b ·2 ^b + R _c ·2 ^c (where the coefficient ( R _a , R _b , R _c ∈ {-1, 0, 1})) and N partial product generators according to the decomposed weights (R _a ·2 ^a , R _b ·2 ^b , R _c ·2 ^c ) Controls the operation of the corresponding partial product generator MBS _i among (MBS).

The weight decomposition unit (WDC) is ^a factor (R _a , R _b , R _c ) and a multiplier of 2 (2 ^a , 2) from the decomposed weights (R _a · 2 ^a , R _b · 2 ^b , R _c · 2 ^c ). ^b, it is possible to pass to the partial product generator (MBS _i) corresponding to the index (a, b, c) of 2 ^c), controlling the operation of partial product generators (MBS _i).

In FIG. 8, as an example, the weight decomposition unit (WDC) is different from the weight decomposition unit (WDC) of Fig. 5, and the weight (w _i ) is divided into three partial products (R _a ·2 ^a , R _b ·2 ^b , R _c ·2). ^Although it is assumed that the decomposition is performed by ^c ), the weight decomposition unit WDC may be set to decompose the weight w _i into two partial products as in FIG. 5.

+1)을 인가받는다. 여기서 네거티브 입력값(

+1)은 입력값(X_i)의 2의 보수값이다. 그리고 N개의 부분곱 생성기(MBS) 각각은 3개의 부분곱 계산기(PP1 ~ PP3)를 포함할 수 있다. 이는 상기한 바와 같이, 가중치 분해부(WDC)가 가중치(w_i)를 3개의 부분곱(R_a·2^a, R_b·2^b, R_c·2^c)으로 분해하는 것으로 가정하였기 때문이다.Each of the N partial product generators (MBS) has an input value (X _i ) and a negative input value (

+1) is authorized. Where the negative input (

+1) is the two's complement value of the input value (X _i ). In addition, each of the N partial product generators MBS may include three partial product calculators PP1 to PP3. This is because, as described above, it is assumed that the weight decomposition unit (WDC) decomposes the weight (w _i ) into three partial products (R _a ·2 ^a , R _b ·2 ^b , R _c ·2 ^c ). .

+1)에 분해된 가중치(R_a·2^a, R_b·2^b, R_c·2^c)에서 계수(R_a, R_b, R_c)를 곱하기 위한 구성으로, 가중치 분해부(WDC)에서 인가되는 계수(R_a, R_b, R_c) 중 대응하는 계수에 응답하여, 입력값(X_i)과 네거티브 입력값(

+1) 및 접지 전원 중 하나를 선택하여 비트 시프터(BSH)로 전달한다.Each of the three partial product calculators (PP1 to PP3) includes a mux (PMX) and a bit shifter (BSH). The mux (PMX) is the input value (X _i ) and the negative input value (

+1) is multiplied by the coefficients (R _a , R _b , R _c ) from the decomposed weights (R _a ·2 ^a , R _b ·2 ^b , R _c ·2 ^c ).WDC In response to the corresponding one of the coefficients (R _a , R _b , R _c ) applied from, the input value (X _i ) and the negative input value (

It selects one of +1) and ground power and delivers it to the bit shifter (BSH).

+1)을 선택함으로써, 네거티브 부분곱을 획득할 수 있도록 한다. 또한, 계수(R_a, R_b, R_c)가 0인 경우에 제로 곱셈을 수행하지 않고, 논리 0을 나타내는 접지 전원을 선택하여 출력하도록 한다. 먹스(PMX)가 네거티브 입력값(

+1) 및 논리 0의 접지 전원을 선택할 수 있도록 구성됨에 따라 도8 의 부분곱 계산기(PP1 ~ PP3)는 1의 보수 연산기(1CMP) 및 제로 곱셈기(ZOP)를 포함하지 않아도 무방하다.As described above, the coefficients R _a , R _b , and R _c may be specified as one of values of -1, 0, and 1. In the partial product calculators PP1 to PP3 of FIG. 8, the mux (PMX) is inverted, unlike FIG. 5, which inverts the input value (X _i ) when the coefficients (R _a , R _b , R _c ) are -1. The applied negative input value (

By selecting +1), it is possible to obtain a negative partial product. In addition, when the coefficients R _a , R _b , and R _c are 0, zero multiplication is not performed, and a ground power supply representing logic 0 is selected and output. The mux (PMX) is the negative input value (

The partial product calculators PP1 to PP3 of FIG. 8 may not include a one's complement operator 1CMP and a zero multiplier ZOP, as it is configured to select +1) and a ground power source of logic 0.

The bit shifter (BSH) receives the selected value from the mux (PMX), and in response to the exponents (a, b, c) applied from the weight decomposition unit (WDC), the applied value is bit-shifted in the upper bit direction. Outputs the product (P1, P2, P3).

The partial product adder (PSUM) receives three partial products (P1 _i , P2 _i , P3 _i ) from each of the N partial product generators (MBS) in the same manner, and adds a bias value (BS) to all of them, Output the output value (Out).

9 shows a block diagram of a digital neuron of an artificial neural network according to another embodiment of the present invention, FIG. 10 shows an example of a detailed configuration of a neuron in the artificial neuron of FIG. 9, and FIG. 11 is a partial product of FIG. An example of a detailed configuration of an adder is shown.

In FIG. 3, as an example, it is assumed that a digital neuron is applied with one two-dimensional input feature map and one two-dimensional weight filter corresponding to the two-dimensional input feature map, and the digital neuron includes one neural element (NE). Shown as

However, as shown in FIG. 1, the artificial neurons of the convolution layers C1, C2, and C3 can receive a 3D input feature map and perform a 3D convolution operation using a 3D weight filter.

Here, the 3D input feature map may be composed of M 2D input feature maps (where M is a natural number), and the M 2D input feature maps are M channels of the 3D input feature map. In this case, the 3D input feature map composed of M 2D input feature maps may be a part of the output feature map output from the previous layer. As an example, the output feature maps output from the previous layer may be composed of L (a natural number of L> M) (32 as an example in FIG. 1) 2D output feature maps. In addition, the artificial neuron may select M two-dimensional output feature maps from the L two-dimensional output feature maps, and may be applied as M input feature maps constituting a three-dimensional input feature map.

In addition, in the same convolutional layer, each of a plurality of artificial neurons may receive a 3D input feature map composed of a different number of 2D input feature maps.

Meanwhile, corresponding to the M channels of the 3D input feature map, the 3D weight filter also has M 2D weight filters. Here, each of the M 2D input feature maps may be output feature maps generated by each artificial neuron of the previous convolutional layer.

Accordingly, FIGS. 9 to 11 illustrate a case in which the digital neuron is configured to receive M 2D input feature maps and M 2D weight filters, and the digital neuron is configured to include M neurons (NE).

Each of the M neural elements NE shown in FIG. 9 has a configuration similar to that of the neural elements NE shown in FIG. 3. However, in the digital neurons shown in Figs. 9 and 10, each of the M neurons (NE) does not add up the bias value (BS), and 2N ((P1 ₁ ,) applied from the N partial product generators (MBS) Only P2 ₁ ), (P1 ₂ , P2 ₂ ), …, (P1 _N , P2 _N )) are summed and the channel calculation value (MO ₁ ~ MO _M ) is output.

In addition, the digital neuron of FIG. 9 adds the M channel operation values MO ₁ to MO _M output from each of the M neurons NE and the bias value BS, and outputs an output value Out. Here, the output value Out is a result of a dot product operation on a partial area of the 3D input feature map by the 3D weight filter, and is an element of the output feature map in 2D.

The digital neuron may then move the 3D weight filter on the 3D input feature map and perform dot product operation again to calculate another element of the 2D output feature map.

In FIG. 10, as an example, a plurality of nerve elements NE is shown to have the same configuration as in FIG. 3. However, the plurality of neuro elements NE may have a configuration according to FIG. 8.

12 shows the result of simulating power consumption of digital neurons according to the present embodiment.

In FIG. 12, power consumption of a plurality of commercial GPUs and digital neurons according to the present embodiment as a general-purpose computing device mainly used for learning and execution of a general artificial neural network is compared. Here, the comparison of the digital neuron according to the present embodiment with a commercial GPU is because the GPU exhibits better performance than the CPU and consumes less power in an artificial neural network that repeatedly performs simple operations.

Referring to FIG. 12, it can be seen that a digital neuron implemented in hardware according to the present embodiment consumes less than 300mW of power, whereas a commercial GPU consumes 100W or more. That is, compared to a commercial GPU, the digital neuron of this embodiment can reduce power consumption to less than 1/300.

Table 4 shows the results of comparing the performance of digital neurons according to the present embodiment with other artificial neurons.

Referring to Table 4, it can be seen that the digital neuron according to the present embodiment exhibits similar accuracy to the method of performing a boost multiplier using a 5-bit integer weight, while gate delay and power consumption are significantly reduced. have. In addition, it can be seen that the power consumption is higher than that of YodaNN artificial neurons using 1-bit integer weights, but the accuracy is remarkably high.

As a result, the digital neuron, the artificial neuron, and the inference engine including the same for the artificial neural network according to the present embodiment minimize the number of partial products because the weight converted to an integer of the specified number of bits excluding some values is used. In addition, the structure of the partial product circuit can be simplified by obtaining a negative partial product by performing a one's complement operation instead of two's complement. In addition, when calculating the channel operation value MO, the sign bit value is separately calculated using the count value NUM_P and then added, thereby simplifying the partial product adder structure. Therefore, high-speed operation is possible, power consumption can be reduced, and a low area can be implemented.

13 shows an example of an artificial neuron capable of varying the size of a weight filter using a digital neuron according to an embodiment of the present invention.

The digital neuron of FIG. 9 receives a 3D input feature map including M 2D input feature maps, and performs a convolution operation or a dot product operation using a 3D weight filter including M 2D weight filters. It was constructed to be able to.

However, the digital neuron of FIG. 9 was not easy to adjust the number of weights included in the weight filter. That is, it was not easy to adjust the size of the weight filter.

If the weight filter is composed of M two-dimensional weight filters with a size of 5 X 5, the digital neurons in Fig. 9 are less than 5 X 5 (for example, 3 X 3) and less than M two-dimensional weight filters. For the dummy weight (e.g. 0) that does not affect the dot product operation, it is converted into M two-dimensional weight filters with a size of 5 X 5 to perform the operation, thereby performing the dot product operation of the input feature map and the weight filter. Can be done.

That is, the digital neuron can easily perform a dot product operation or a convolution operation on a weight filter less than a specified size when designing digital hardware. However, a dot product operation or a convolution operation cannot be performed on a weight filter exceeding the specified size.

In order to prevent such a problem, when the size of a weight filter that can be computed in advance when designing a digital neuron is greatly expanded, not only a number of unnecessary calculations are performed, but also the size and power consumption of the digital neuron are increased.

In FIG. 13, a structure of an artificial neuron capable of improving the operation speed when the operation is performed using a weight filter exceeding the size specified for the digital neuron, and the operation is performed using a weight filter less than the specified size. Illustrated conceptually.

Referring to FIG. 13, the artificial neuron according to the present embodiment includes an input value extractor (INEX), a plurality of digital neurons (DN), an output adder (AD), at least one input selection mux (Mux1), and an output selection mux ( Mux2).

The input value extracting unit INEX extracts an input value of a specified size at a predetermined location from the input feature map. In this case, the input value extracting unit INEX may extract input values of different positions in response to the mode signal MD, and the mode signal MD may designate a basic mode and an extended mode.

Here, the basic mode is a mode for performing an operation using a weight filter less than a specified size in which each of a plurality of digital neurons (DN) can operate, and the extended mode is a mode for performing an operation using a weight filter exceeding the specified size. Mode.

In FIG. 13, as an example, it is assumed that the size of a designated weight filter of a digital neuron (DN) is M two-dimensional weight filters having a size of 5 X 5. Accordingly, the basic mode may be set when the weight filter includes a size of 5 X 5 or less and M or less 2D weight filters according to a size designated for the digital neuron (DN).

However, when the size of the weight filter exceeds the size of 5 X 5 or includes more than M two-dimensional weight filters, an extension mode is specified.

Here, it is assumed that the weight filter is composed of M two-dimensional weight filters of size 5 X 5 and the case of M two-dimensional weight filters of 7 X 7 size is assumed.

When the mode signal MD is designated as the basic mode, the input value extraction unit INEX extracts two input value sets having a size corresponding to the size of the weight filter from the input feature map.

In the basic mode, the input value extracting unit INEX extracts two sets of input values each corresponding to the size of the weight filter.

Referring to FIG. 13, in the basic mode, the input value extracting unit INEX uses the input values of the first area IN _T1 and the second area IN _T2 of the input feature map as a first set of basic input values. Then, the input value of the second area IN _T2 and the input value of the fourth area IN _T4 are extracted as a second basic input value set.

A third region (IN _T3 but extracts any entry in the expansion mode part extracting the input value and the input value of the second area (IN _T2) of the first area (IN _T1) of the input feature map as a first expansion set of input values, and ) And the input value of the fourth region IN _T4 are extracted as a second extended input value set.

That is, the same input values are extracted from the first basic input value set and the first extended input value set, while different input values are extracted from the second basic input value set and the second extended input value set.

The extracted first set of basic input values and the first set of extended input values are applied to a corresponding first digital neuron (DN1) among a plurality of digital neurons, and the second set of basic input values and the second set of extended input values correspond. It is applied to the second digital neuron (DN2).

In the basic mode, the first and second digital neurons (DN1, DN2) each perform a dot product operation using the applied first set of basic input values, the second set of basic input values, and a designated weight filter, and output values (Out ₁ , Out). ₂ ) is printed. Here, the weight filters of the first and second digital neurons DN1 and DN2 are the same weight filters.

That is, in the basic mode, the first and second digital neurons DN1 and DN2 simultaneously perform an operation in which the weight filter position is moved in a convolution operation in which the weight filter moves on the input feature map and performs the dot product operation. That is, in a convolution operation composed of a plurality of inner product operations, two inner product operations can be performed in parallel using two digital neurons DN1 and DN2, thereby doubling the operation speed.

In Fig. 13, as an example, it is assumed that the artificial neuron includes two digital neurons (DN1, DN2), so the calculation speed is doubled, but if the number of digital included in the artificial neuron increases, the calculation speed can also be proportionally improved. have.

Meanwhile, in the extended mode, each of the first and second digital neurons DN1 and DN2 performs a dot product operation using a first set of extended input values, a second set of extended input values, and a corresponding region of a designated weight filter, and output values ( Out ₁ ) is output.

In the extended mode, since the size of the weight filter is larger than the size that the digital neurons (DN1, DN2) can calculate, the dot product operation required by only one digital neuron cannot be performed. Accordingly, in this embodiment, a plurality of digital neurons (DN1, DN2) receive input values and weights of a corresponding region of a large-sized weight filter to perform a dot product operation, respectively, so that the digital neurons (DN1, DN2) It is possible to perform an operation even on a weight filter of a size larger than the size that can be calculated.

In the above, it is assumed that the size of the weight filter that each digital neuron (DN1, DN2) can calculate is 5 X 5 XM, so each digital neuron (DN1, DN2) calculates 25 XM input values and weights. Can be done. Therefore, two digital neurons DN1 and DN2 can perform calculations on 50 X M input values and weights.

Since it is assumed that the weight filter applied in the extended mode is 7 X 7 XM (= 49 XM), artificial neurons using 2 digital neurons (DN1, DN2) of 5 X 5 XM size are easily used. It is possible to perform a dot product operation on the weight filter.

However, parallel operation in basic mode must be output as individual output values (Out ₁ , Out ₂ ), whereas in extended mode, output values of multiple digital neurons (DN1, DN2) must be integrated and output as a single output value (Out ₁ ). do.

Accordingly, the artificial neuron of FIG. 13 further includes an output adder ADD and an output selection mux2. The output adder ADD sums the output values of a plurality of digital neurons DN1 and DN2 in the extended mode and outputs the sum to the output selection mux2.

In response to the mode signal MD, the output selection Mux2 selects the output value output from the first digital neuron DN1 in the basic mode, while outputting the sum value output from the output adder ADD in the extended mode. .

Meanwhile, as shown in FIG. 13, the first set of basic input values and the second set of basic input values overlap the input values of the second area IN _T2 . In addition, the second basic input value set and the second extended input value set overlap the input value of the fourth area IN _T4 , while the input value of the third area IN _T3 is included only in the second extended input value set. .

When the overlapping input values and different input values are individually applied to a plurality of digital neurons (DN1 and DN2), a plurality of data lines must be arranged on the artificial neuron. In addition, since a set of input values is output from each of the M channels, the number of data lines that must be provided to transmit redundant input values is very large, which increases the size and power consumption of artificial neurons.

Accordingly, in this embodiment, a plurality of input selection muxes (Mux1) are provided, so that the input values overlapping in the second basic input value set and the second extended input value set and the differentiated input value are selectively selected as the second digital neuron DN2. To be applied.

Here, the at least one input selection mux1 may select one of a redundant input value and a differential input value of the second basic input value set and the second extended input value set in response to the mode signal MD.

14 shows a schematic configuration of an inference engine of an artificial neural network according to another embodiment of the present invention.

14 shows an example of an inference engine for a convolutional neural network. In particular, the inference engine of FIG. 14 includes one convolution engine unit CEN, one fully connected engine unit FEN, and a memory MEM. As shown in FIG. 1, the convolutional neural network may include a plurality of convolutional layers C1, C2, and C3 and a plurality of fully connected layers FC1 and FC2. However, implementing multiple convolution layers (C1, C2, C3) and multiple fully connected layers (FC1, FC2) as separate circuits is very inefficient in terms of size, power consumption, and manufacturing cost.

Accordingly, in FIG. 14, one convolution engine unit (CEN) for performing operations of a plurality of convolutional layers performing similar operations and one fully connected engine unit (FEN) for performing operations of a plurality of fully connected layers. Including, the convolutional neural network can be implemented with digital hardware.

In FIG. 14, for example, as shown on the left, it is assumed that the artificial neural network includes two convolutional layers C1 and C2 and three fully connected layers FC1, FC2, and INF.

First, the memory MEM stores input data (or input image) applied to the convolutional neural network, a weight filter for a plurality of convolutional layers (C1, C2) and a plurality of fully connected layers (FC1, FC2, INF). .

The convolution engine unit (CEN) includes an input value acquisition unit (INOB), a convolution filter bank buffer (FB1), an input mux (MuxIN), an input selection mux (MuxDI), a demux, and an output feature map buffer (OFMB). ) And at least one digital neuron (DN1).

The input Muxin is an input part of an artificial neural network, receiving input data stored in the memory MEN or an output value of the convolution engine unit CEN, and in response to the layer selection signal SelINL, it receives either an input image or an output value. It selects and outputs an input value, which is an element of the input feature map, to the input value acquisition unit INOB.

Here, the layer selection signal (SelIN _L ) is a signal for selecting an initial input value of the convolution engine unit (CEN) in an artificial neural network including a plurality of convolutional layers (C1, C2), and is an initial layer of the artificial neural network (here This is a signal for discriminating whether it is the first convolution layer C1) or a layer after the convolution layers C1 and C2 (here, the fully connected layers FC1, FC2, and INF).

As described above, when the artificial neural network includes two convolution layers (C1, C2), the first convolution layer (C1) receives input data, whereas the second convolution layer (C2) receives the first It may be configured to receive the output value of the convolution layer C1.

Based on the above-described structure, the input muxin responds to the layer selection signal SelIN _L , and if it is the initial layer C1, selects input data from the memory MEM and outputs the input data to the input value acquisition unit INOB. On the other hand, if it is not the convolution layers C1 and C2, the output value of the convolution engine unit CEN is selected and output to the input value acquisition unit INOB.

Here, when the convolution layers C1 and C2 are not, the output value of the convolution engine unit CEN may be output as a predetermined set value or an initialized value. In other words, if the input muxin is not the convolutional layer (C1, C2), it outputs the initialized value to the input value acquisition unit (INOB) and initializes the input value acquisition unit (INOB) to a predetermined initialization value. I can.

The input value acquisition unit INOB receives input data from the input Muxin, configures and stores an input feature map, extracts the input value included in the designated area from the stored input feature map, and uses the input selection MuxDI. Print.

As described above, in order to perform a convolution operation, the convolution layer must perform dot product operation while a designated weight filter is moved on an input feature map.

This should be transferred to the digital neuron (DN1) by selecting the input value of the position corresponding to the weight filter among the plurality of input values of the input feature map stored in the input value acquisition unit (INOB) while moving. Very difficult to implement.

However, in this embodiment, the input value acquisition unit (INOB) is implemented as a plurality of shift registers as shown in FIG. 14, and a plurality of input values of the input feature map are sequentially moved to a designated position. Accordingly, the input value at the same position is always extracted and output as an input selection MuxDI.

That is, it is possible to implement the input value acquisition unit INOB in hardware very easily using the shift register.

In addition, the input value acquisition unit INOB is initialized by storing an initialization value applied from the input muxin when the layer to be currently performed is not the convolutional layers C1 and C2.

In response to the input data selection signal SelDI _L , the input selection MuxDI selects one of an input value applied from the input value acquisition unit INOB or an output value applied from the output feature map buffer OFMB. Output as (DN1).

Here, the input data selection signal SelDI _L is a signal for determining whether or not the convolutional layer to which the convolution engine unit CEN currently performs a convolution operation in the artificial neural network is the initial convolutional layer C1.

The input selection MuxDI outputs an input value applied from the input value acquisition unit INOB to a digital neuron (DN1) if it is the initial convolution layer (C1) among a plurality of convolution layers (C1, C2) of the artificial neural network. do.

However, if it is not the initial convolution layer C1, the value of the output feature map applied from the output feature map buffer OFMB is output as an input value to the digital neuron DN1. That is, the output feature map obtained from the previous convolution layer is transferred to the digital neuron DN1 as an input feature map.

The convolution filter bank buffer FB1 receives and temporarily stores the weight filter stored in the memory MEN. At this time, the convolution filter bank buffer (FB1) may receive and store a plurality of weight filters corresponding to each of a plurality of convolution layers (C1, C2) at once, but it corresponds to an operation performed by the current convolution engine (CEN). A weight filter for a convolutional layer may be applied and stored.

The digital neuron DN1 is an artificial neuron shown in FIG. 9 and outputs a plurality of input values applied from the input selection MuxDI and a weight filter applied from the convolution filter bank buffer FB1 by dot product.

In response to the output data selection signal SelDO _L , the Demux outputs an output value output from the digital neuron DN1 to the output feature map buffer OFMB or the fully connected engine FEN.

Here, the output data selection signal (SelDO _L ) is different from the input data selection signal (SelDI _L ).In the artificial neural network, the convolutional layer to which the convolution engine unit (CEN) currently performs convolution operation is the final convolution layer or not. It is a signal to discriminate.

Demux (Demux) is the final convolution layer (C2) of the plurality of convolution layers (C1, C2), outputs the output value of the digital neuron (DN1) to the fully connected engine (FEN). However, if it is not the final convolutional layer C2, the output value of the digital neuron DN1 is outputted to the output feature map buffer OFMB.

In addition, the output of the demux is applied as an input muxin when the layer on which the current operation is performed is not the convolutional layers C1 and C2.

The output feature map buffer OFMB sequentially accumulates output values output from the Demux and stores them as an output feature map, and outputs the stored output values as an input selection MuxDI.

The convolution engine (CEN) configured as described above receives the input image stored in the memory (MEM) as an input feature map for the initial convolution layer among the plurality of convolution layers of the feature extraction unit, extracts the input value, and extracts the digital neuron (DN1). ), and stores the output value output from the digital neuron DN1 in the output feature map buffer (OFMB).

In addition, for the convolutional layer in the intermediate stage, the output feature map is applied from the output feature map buffer OFMB as an input feature map and is stored in the output feature map buffer OFMB.

However, for the final convolutional layer, the output value output from the digital neuron (DN1) is output to the fully connected engine (FEN).

That is, a feature extractor including a plurality of convolution layers C1 and C2 may be implemented with one convolution engine CEN.

At this time, the convolution engine CEN may further include a pooling unit (not shown) for sub-sampling the output feature map stored in the output feature map buffer OFMB in a known manner. The pooling unit may be configured to perform an average value pooling or a max pooling operation, for example.

In addition, although it is illustrated that the output feature map buffer OFMB is included in the convolution engine CEN, the output feature map may be configured to be stored in the memory MEM in some cases.

Meanwhile, the fully connected engine FEN may include an input inference selection MuxFI, an input register FIRG, an output register FORG, a fully connected filter bank buffer FB2, and a digital neuron DN2.

The input inference selection MuxFI is similar to the input selection MuxDI of the convolution engine (CEN) in response to the connection layer selection signal (SelFI _L ) and the output value or output register (FORG) applied from the convolution engine (CEN). Select the output value stored in) and transfer it to the input register (FIRG).

Here, the connection layer selection signal SelFI _L is a signal applied to distinguish whether it is the initial fully connected layer (FC1) or the remaining fully connected layers (FC2, INF) among the plurality of fully connected layers (FC1, FC2, INF) of the classification unit. to be.

In response to the connection layer selection signal SelFI _L , the input inference selection MuxFI selects an output value applied from the convolution engine CEN and transfers it to the input register FIRG if it is the initial fully connected layer FC1. . However, in the remaining fully connected layers FC2 and INF, the output value stored in the output register FORG is selected and transferred to the input register FIRG.

The input register (FIRG) receives the output value transmitted from the input reasoning selection MuxFI, temporarily stores it, and outputs it to the digital neuron (DN2), and the output register (FORG) temporarily stores the output value output from the digital neuron (DN2). And output as input reasoning selection MuxFI. In addition, the fully connected filter bank buffer FB2 receives and temporarily stores the weight filter stored in the memory MEN.

The digital neuron DN2 receives the output value applied from the input register FIRG and the weight filter stored in the fully connected filter bank buffer FB2, performs a predetermined operation, and outputs the output value. In this case, the output value output from the digital neuron DN2 may be an inferencr output.

Here, the fully connected engine FEN may further include a demux that selects to output an output value output from the digital neuron DN2 to the output register FORG or output to the outside.

In this case, the convolution engine (CEN) and the fully connected engine (FEN) may have almost the same structure. However, the reason why the inference engine separates the convolution engine (CEN) and the fully connected engine (FEN) and has a large difference between the size of the weight filter used in the convolution layer and the weight filter used in the fully connected layer is generally very different. Because it is large, it is very inefficient to implement a fully connected layer using the convolution engine (CEN).

That is, in FIG. 14, the digital neuron DN1 of the convolution engine (CEN) and the digital neuron DN2 of the fully connected engine (FEN) may both be implemented as artificial neurons as shown in FIG. 9, but the weight filter and the input feature map Since the size difference is large, it is desirable to implement it separately. In particular, the digital neuron (DN1) of the convolution engine (CEN) uses a 3D weight filter, while the digital neuron (DN2) of the fully connected engine (FEN) mostly uses a 1D weight filter.

Accordingly, in this embodiment, the digital neuron (DN1) of the convolution engine (CEN) is composed of the artificial neuron of Fig. 9, and the digital neuron (DN2) of the fully connected engine (FEN) is a fully connected artificial neuron that is implemented separately. have.

In FIG. 14 above, it is assumed that the convolution engine (CEN) includes one digital neuron (DN1), but each of the plurality of convolutional layers of the artificial neural network of FIG. 14 may use weight filters of different sizes. have. In this case, the size of the weight filter may be larger than the size of the weight filter designated by the digital neuron DN1.

Accordingly, the inference engine for the artificial neural network of FIG. 14 may include the artificial neuron of FIG. 13 instead of the digital neuron DN1. For example, in the inference engine of FIG. 14, the convolution engine (CEN) may include a plurality of digital neurons as shown in FIG. 13. In addition, the input value acquisition unit INOB of FIG. 14 may be configured to extract input values of different positions in response to the mode signal MD, like the input value extracting unit INEX of FIG. 13. In addition, it includes a plurality of input selection muxes (MuxDI) between the input value acquisition unit (INOB) and a plurality of digital neurons, and an adder (AD) and at least one output selection mux between a plurality of digital neurons and Demux. It can be configured to include (Mux2).

That is, two or more dot product operations are performed in parallel for weight filters within a specified size according to the size of the weight filter of the convolution layer corresponding to the operation that the convolution engine (CEN) should perform, and weight filters exceeding the specified size are performed. For this, it is divided so that the dot product operation can be performed.

In some cases, the fully connected engine (FEN) may also include a plurality of artificial neurons, similar to the convolution engine (CEN), and may be configured to perform dot product operations in a parallel or split method according to the size of the weight.

The method according to the present invention may be implemented as a computer program stored in a medium for execution on a computer. Here, the computer-readable medium may be any available medium that can be accessed by a computer, and may also include all computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, and ROM (Read Dedicated memory), RAM (random access memory), CD (compact disk)-ROM, DVD (digital video disk)-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

The present invention has been described with reference to the embodiments shown in the drawings, but these are merely exemplary, and those of ordinary skill in the art will understand that various modifications and equivalent other embodiments are possible therefrom.

Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

Claims (12)

Digital implemented by digital hardware that performs dot product operation by receiving an input vector including a plurality of input values in an integer format having a predetermined number of bits and a weight vector including a plurality of weights in an integer format having a predetermined number of bits. For neurons,
The digital neuron includes a neuron, and the neuron
A plurality of values expressed as a product of a coefficient (R) having at least one value of -1, 0, 1 and a multiplier of 2 (2 ⁿ ) having a corresponding weight applied from the weight vector and the weight of the applied integer format A plurality of weight decomposition units that decompose into the sum of the partial products of R·2 ⁿ and output a control signal according to a coefficient (R) and an exponent (n) of the decomposed partial products;
A plurality of partial product generators for receiving a corresponding input value from the input vector, and outputting a plurality of partial products by bit shifting the input value in a direction of an upper bit by an index (n) in response to the control signal; And
A partial product adder for summing the plurality of partial products in parallel and outputting a channel operation value; Including,
Each of the plurality of partial product generators
A digital neuron that additionally performs a one's complement operation or a zero conversion operation for the plurality of partial products according to the coefficient R.
delete The method of claim 1, wherein the digital neuron
A negative partial product counter for obtaining a count value by counting the number of times a one's complement operation is performed by the plurality of partial product generators; Including more,
The partial product adder is
A digital neuron that calculates the channel operation value by summing the count values together with the plurality of partial products. The method of claim 3, wherein the partial product adder is
A digital neuron that calculates the channel operation value by summing the count value and the value excluding the value of the sign bit from the plurality of partial products, and adding the two's complement of the count value to the sign bit to the summation result. The method of claim 4, wherein each of the plurality of partial product generators
It includes a plurality of partial product calculators that are driven according to the control signal to calculate and output each partial product,
Each of the plurality of partial product calculators
A bit shifter for bit shifting the input value by the exponent n;
A one's complement operator for outputting the partial product by performing a one's complement operation on the output value of the bit shifter when the coefficient R is -1; And
A zero multiplier for converting the partial product to zero in response to the control signal when the coefficient R is 0; Digital neurons further comprising. The method of claim 4, wherein the partial product adder is
It is composed of a number of stages each including a number of full adders,
The first stage among the plurality of stages receives the plurality of partial products, groups and adds values of the same bit of the partial products of different preset numbers,
The remaining stages are added by grouping the addition values of the previous stage,
The final stage is a digital neuron that calculates the channel operation value by adding the count value to the lower bit of the addition value of the previous stage. The method of claim 6, wherein the partial product adder is
A two's complement calculator that receives the count value and calculates a two's complement value; Including more,
The final stage is a digital neuron that extends the sign bit of the channel operation value by adding the count complement value to the upper bit of the addition value of the previous stage. The method of claim 1, wherein the digital neuron
A corresponding two-dimensional input feature map among a number of two-dimensional weight filters having a plurality of weights of the three-dimensional weight filter and a two-dimensional input feature map having a plurality of input values of the three-dimensional input feature map Including a plurality of neurons each approved,
A channel adder for outputting an output value of the digital neuron by adding the channel operation value output from each of the plurality of neurons and a predetermined bias value; Digital neurons further comprising. The method of claim 1, wherein the weight is
In order to reduce the number of items of the partial product, a digital neuron that is applied by being substituted with a designated adjacent integer so that a predetermined integer is excluded. A nerve implemented by digital hardware that performs dot product operation by receiving an input vector including a plurality of input values in an integer format having a predetermined number of bits and a weight vector including a plurality of weights in an integer format having a predetermined number of bits. In the digital neuron containing the device,
The nerve element
A plurality of values expressed by the product of a coefficient (R) having at least one value of -1, 0, and 1 and a multiplier of 2 (2 ⁿ ) having a corresponding weight applied from the weight vector and the weight of the applied integer format A plurality of weight decomposition units that decompose into a sum of partial products (R·2 ⁿ ) of;
A value corresponding to the coefficient R is selected by receiving a corresponding input value from the input vector and a voltage corresponding to the negative input value and the 0 value of the input value, and the selected value is in the direction of the upper bit by the index (n). A plurality of partial product generators for bit shifting to and outputting a plurality of partial products, respectively; And
A partial product adder for summing the plurality of partial products in parallel and outputting a channel operation value; Including,
Each of the plurality of partial product generators
A digital neuron that additionally performs a one's complement operation or a zero conversion operation for the plurality of partial products according to the coefficient R.
The method of claim 10, wherein each of the plurality of partial product generators
Includes a plurality of partial product calculators that calculate and output each partial product,
Each of the plurality of partial product calculators
A mux for receiving a voltage corresponding to the input value and a negative input value and a zero value, which are two's complement of the input value, and selecting and outputting a value corresponding to the coefficient R; And
A bit shifter for bit shifting a value selected and applied from the mux by the index n; Digital neurons containing. The method of claim 10, wherein the weight is
In order to reduce the number of items of the partial product, a digital neuron that is applied by being substituted with a designated adjacent integer so that a predetermined integer is excluded.

Images