KR20200050895A - Log-quantized mac for stochastic computing and accelerator comprising the same - Google Patents

Abstract

The present invention relates to a log-quantized multiply-accumulator for stochastic calculation and an accelerator including the same. According to an embodiment of the present invention, the multiply-accumulator comprises: an input providing part providing input data; a weight providing part that receives a weight and outputs a log-quantized weight; a log-to-linear converter that receives the log-quantized weight and outputs a linearly transformed weight; an inverse counter that outputs a window width from the linearly converted weight; a 1 counter counting a value of 1 in the window width from the number of stochastics corresponding to the input data; and an accumulator that accumulates and outputs the value output by the 1 counter.

Description

Log-QUANTIZED MAC FOR STOCHASTIC COMPUTING AND ACCELERATOR COMPRISING THE SAME}

The present technology relates to a MAC performing a stochastic operation using a log-quantized weight value and an accelerator including the same.

Stochastic computing (stochastic computing) is a computing method using a method of expressing a number using an appearance frequency of 1 in a bitstream, which is lower cost than digital implementations based on 2's complement. It has many advantages such as low power consumption, high error-resistance characteristics, and gradual precision. Stochastic computing has recently been shown to be effective for accelerating deep neural networks (DNNs) as well as other applications such as low-power image processing.

 Stochastic computing is useful in the context of approximate computing to trade-off processing time and energy consumption and resulting quality depending on the environment, input or computing situation (e.g., different layers of DNN). Provide a framework. This run-time adjustment is very natural and does not introduce overhead in stochastic computing.

On the other hand, in the human brain, neurons connect to form a network, in the process of learning, reasoning and thinking. Neural networks are formed by imitating the learning activities. The components of the neural network may be neurons and synapses, and synapses are the parts where neurons and neurons are connected, and the strong and weak degree of synaptic connection between neurons is called weight.

Learning a neural network is when a network is created and given data that you want to learn, when a value corresponding to the desired data is entered, and multiplied by the weight value until the desired value is changed and output. As far as learning is done, as a result, the process of learning is a process of obtaining weights.

The neural network learning and inference is performed using an accelerator, because performing a learning program (and inference program) called a neural network performs a number of parallel operations. That is, a weighted neuron between a neuron and a neuron can be expressed as a weighting matrix, and multiplied by an input vector and a weighting matrix to obtain an output vector, but the number of operations performed is large.

One of the most important problems of stochastic computing is that run time and energy consumption increase exponentially with respect to precision. Stochastic computing requires a bit stream of length 2 ^N to achieve precision similar to N-bit binaries, which is why the exponential complexity of SC to achieve high precision affects both computational and storage requirements, However, it becomes a serious problem to apply SC to large problems such as large-scale SC-based DNN. Bit-parallelism is one of the ways to reduce latency, but it is expensive.

The multiply-accumulator according to the present embodiment is provided with an input providing unit providing input data, a weight providing unit receiving weight input and outputting quantized weights to a log, and a log quantized weight , A log-to-linear converter that outputs linearly converted weights, an inverse counter that outputs a window width from linearly converted weights, a 1 counter that counts a value of 1 in the window width from the number of stochastics corresponding to the input data, and 1 Includes an accumulator that accumulates and outputs the values output by the counter.

The matrix vector multiplier according to the present embodiment includes an input providing unit providing a plurality of input data vectors, a weight providing unit receiving a weight input and outputting quantized weights to a log, and quantized with a log A log-to-linear transformer that receives weights and outputs linearly converted weights, an inverse counter that outputs window widths from linearly converted weights, and 1 in the window width in the number of stochastics corresponding to each input data vector. It includes a plurality of 1 counters for counting values and a plurality of accumulators for accumulating and outputting values output by 1 counter.

According to the present embodiment, a multiplication and accumulator having high energy efficiency with a small area and an accelerator using the same are provided.

1 is a diagram showing an outline of a multiplication and accumulator using log quantization according to the present embodiment.
FIG. 2 (a) is a schematic diagram for explaining the operation of the stochastic number generator, and FIG. 2 (b) is a diagram for explaining the operation outline of the stochastic number generator according to bit parallelization, and FIG. 2 (c) is a bit. A diagram for explaining another example of the operation of the stochastic number generator 120 according to parallelization.
FIG. 3 (a) is a diagram illustrating a case in which the window width provided in 1 counter operating according to bit parallelization is 8, and FIG. 3 (b) shows a case in which window width provided in 1 counter operating according to bit parallelization is 2 It is an illustrative drawing,
4 is a block diagram showing the outline of one counter according to the present embodiment.
5 is a block diagram showing an outline of an accelerator including a multiplier and an accumulator according to the present embodiment.
FIG. 6 shows the multiplication error expressed as the mean square error of the stochastic computing accelerator.
FIG. 7 (a) is a view illustrating an area of an accelerator, and FIG. 7 (b) is comparing power consumption.
FIG. 8 is a diagram showing the result of recognition test on a target neural network fine-tuned 5000 times, and FIG. 8 (a) is a 10-class image classification result, and FIG. 8 (b) is a 1000 class image classification result.

Hereinafter, this embodiment will be described with reference to the accompanying drawings. 1 is a diagram showing an outline of a multiply and accumulator (MAC) 100 using log quantization according to the present embodiment. Referring to FIG. 1, the multiplication and accumulator 100 according to this embodiment outputs an input providing unit 110 providing an input (x) and a stochastic number corresponding to the input (x). Stochastic number generator (120, SNG, stochastic number generator) and 1 counter (130, 1s cnt) for counting 1 included in the number of stochastics, and a weight providing unit (150, for outputting a logarithm) weight provider), a log-to-linear converter (160) that linearly converts the input log quantized number, and an inverse counter (170, down-cnt) and weight (~) subtracted from a predetermined number. and an XOR gate that computes and outputs the sign of the multiplication result from w) and input (x).

The input providing unit 110 provides an input (x) provided to a neural network (not shown). As an example, the input (x) may be a binary component of an input vector provided to a neural network. In one embodiment, the input providing unit 110 outputs the input x provided outside the multiplier and accumulator 100 according to the present embodiment.

FIG. 2A is a schematic diagram for explaining the operation of the stochastic male generator 120. Referring to FIG. 2, the number of stochastic generators 120 forms the number of stochastics corresponding to the probability of the input (x) and outputs in the form of a bit-stream. In one embodiment, assuming that x is (0.5) dec in decimal and 16 bits, the stochastic number generator 120 sets x3 to 1, x2, x1, and x0 to 0, and select FSM (selector FSM, Finite). State Machine) to control the multiplexer (MUX) to control the output of the bit stream (1010 1010 1010 1010) bin.

In another embodiment, when the value of x is (0.625) dec, since 0.625 = 0.5 + 0.125, the stochastic number generator 120 is set to x3 = x1 = 1, and x2 = x0 = 0. The multiplexer (MUX) is controlled through the selected FSM to output the bit stream (1011 1010 1011 1010) bin.

사이클로 수행된다. 도 2(b)로 예시된 실시예에서, N이 총 16 비트이고, n은 비트 병렬화 매개변수이다. 비트 병렬화가 2² (n=2) 만큼 이루어진 바, 총 4 사이클로 수행된다. FIG. 2 (b) is a diagram for explaining the operation overview of the stochastic number generator 120 according to bit parallelization. Referring to Figure 2 (b), according to the bit parallelization method 1 counter 130

Cycle. In the embodiment illustrated in Figure 2 (b), N is a total of 16 bits and n is a bit parallelization parameter. Bit parallelism is 2 ² (n = 2) As it is made, it is performed in a total of 4 cycles.

1 counter 130 is performed in a total of 4 cycles, and the stochastic number generator 120 divides 16 bits of x into 4 blocks of b1 to b4 each block having 4 bits. The first three bits of each block are the same in the order of x3, x2, and x3, as indicated by the dotted line. The last bit is selected from x1 and x0 by the selection FSM. Accordingly, the set values of x3 and x2 can be written to the three bits of the four blocks b1, b2, b3, and b4 as they are.

For example, when x is (0.5) dec in decimal, the stochastic number generator 120 sets x3 to 1, x2, x1 and x0 to 0, and multiplexes through a selector FSM (Finite State Machine). By controlling the firearm (MUX), the first block (b1) is (1010) bin, the second block (b2) is (1010) bin, the third block (b3) is (1010) bin, and the fourth block (b4) is Print (1010) bin.

As another example, when x is (0.625) dec in decimal, the stochastic number generator 120 sets x3 and x1 to 1 and x2 and x0 to 0. The first block (b1) is (1011) bin, the second block (b2) is (1010) bin, and the third block (b3) by controlling the multiplexer (MUX) through the selector FSM (Finite State Machine). Prints (1011) bin, and the fourth block (b4) prints (1010) bin.

2 (c) is a diagram for explaining another example of the operation of the stochastic number generator 120 according to bit parallelization. Referring to FIG. 2 (c), an example in which x is 16 bits in total and 1 counter 130 is performed in 2 ⁿ (n = 1) = 2 cycles is illustrated. Since 1 counter 130 is performed in 2 cycles in total, the stochastic number generator 120 divides 16 bits of x into 2 blocks of b1 and b2, and each block has 8 bits.

The first seven bits of each block are the same in the order of x3, x2, x3, x1, x3, x2, x3 as indicated by the dotted line. The last bit is selected from x0 and 0 by the selection FSM. Accordingly, the set x3, x2 and x1 values can be written to the seven bits of the two blocks b1 and b2 as they are.

The embodiment illustrated in FIG. 1 illustrates that the stochastic number generator 120 forms the corresponding stochastic number according to the input data (x) value and provides it to the 1 counter 130. However, in the block formed by dividing the number of stochastics as in the embodiment illustrated in FIGS. 2 (b) and 2 (c), values determined according to the bit positions in the blocks are located.

Any one of the 16 bits may be set to 0. That is, x3 is located in the bit stream with a probability of 0.5, x2 is located in the bit stream with a probability of 0.25, x1 is located in the bit stream with a probability of 0.125, and x0 is located in the bit stream with a probability of 0.0625. Even if all of them are combined, the probability does not become 1, so one bit out of a total of 16 bits is set to 0.

은 아래의 수학식 1로 표시될 수 있다. 일 예로, 로그는 밑(base)이 2인 로그일 수 있다.The weight providing unit 150 receives the weight of the neural network (not shown), quantizes it into a log, and outputs it. If the weight provided to the weight providing unit 150 is w, the weight providing unit 150 logs a weight w that is a real number and rounds it up and outputs it. Value output by the weight providing unit 150

Can be represented by Equation 1 below. As an example, the log may be a log whose base is 2.

(round (): rounding function, w: real weight)

의 관계를 데이터 표시를 예시한다.Table 1 shows the decimal real weight

The relationship of data is illustrated.

For example, if the real weight is 0.25, the sign bit is 0, and if the sign is calculated by taking the base 2 logarithm, the value is (2) _dec in decimal and (0010) _bin in binary.

는 0보다 크다. 부호 비트를 포함하는 16비트로 표현될 수 있는 최소한의 단위는

이고, 그때 가중치 제공부(150)의 출력

는 15이다. w가 0인 경우를 제외하고

는 [1,15]의 범위 내에 있으므로, 4 비트의 언사인드(unsigned) 정수로

를 표시할 수 있다. 따라서, 가중치 제공부(150)가 출력하는 로그 양자화된 가중치

는 1 비트의 부호 비트와 크기(magnitude)를 표시하는 4 비트를 포함하는 총 5 비트로 표시된다.Since the real weight is in the range of decimal (-1,1), its absolute value is less than 1. Therefore, the logarithm is rounded to the absolute value of the weight excluding zero

Is greater than 0. The minimum unit that can be expressed in 16 bits including a sign bit is

And then the output of the weight providing unit 150

Is 15. except when w is 0

Is within the range of [1,15], so as a 4-bit unsigned integer

Can be displayed. Therefore, the log quantized weight output by the weight providing unit 150

Is represented by a total of 5 bits including 1 bit sign bit and 4 bits indicating magnitude.

,

등의 2의 거듭제곱으로 표시되며, 2보다 정밀한 밑(base)을 선택하여 가중치를 양자화 하면 뉴럴 네트워크 인식의 정밀도를 개선할 수 있다.As in the illustrated embodiment, the weight is quantized on a logarithmic scale, and in deep neural networks, the base 2 of the log may be taken to obtain floating point level accuracy. In another embodiment,

,

It is represented by a power of 2, etc., and the precision of neural network recognition can be improved by quantizing weights by selecting a base that is more precise than 2.

를 제공받고, 이를 선형 이진수로 변환하여 출력한다. Hereinafter, it is assumed that the precision of 16-bit data including 1-bit sign bit and 15 bits indicating magnitude. However, this is for convenience of description and is not intended to limit the scope of the present invention. Log-to-linear converter 160 is a binary weight quantized to logarithmic

Is provided, converted to a linear binary number, and output.

For example, if w = 0.25 and the log quantized value is 0 0010, the log-to-linear converter 160 output value is displayed as a sign bit + 15 bit decimal part absolute value (0 010 0000 0000 0000) _bin do. In addition, when w = 0.0125 (= 1/8), the log quantum value is 0 0011, and the log-to-linear converter 160 output value is (0 001 0000 0000 0000) _bin . As another example, when w = 0.0625 (= 1/16), the log quantum value is 0 0100, and the log-to-linear converter 160 output value is (0 000 1000 0000 0000) _bin . As can be seen from the above, since the weight value converted to linear is a log-quantized value converted to linear, only one digit of the multiple bits representing the weight has "1".

In one embodiment, as described above, the real weight of 16 bits is taken as a log with a base 2 and quantized to 5 bits including a sign bit. The log-to-linear converter 160 receives a 5-bit quantized weight and outputs a 15-bit linearly converted weight including the sign bit to the inverse counter 170.

The inverse counter 170 receives a 15-bit linearly converted weight output from the log-to-linear converter 160, calculates a window width corresponding to the linearly converted weight, and outputs it to the 1 counter 130 do. In one embodiment, the window width w may be a number obtained by multiplying the number of bits by a weight linearly converted. For example, if the value corresponding to the linearly converted weight is 010 0000 0000 0000 excluding the sign bit, it is .25, which is a prime number. That is, it means that the linearly converted weight value contains 1 with a probability of 0.25. Therefore, if the number of bits representing the weight is 16, the weight includes four "1s", and the window width at this time corresponds to four. As another example, the logarithmic quantum value is 0 0011 when the linearly converted weight value w = 0.0125 (= 1/8), and the output value of the log-to-linear converter 160 is (0 001 0000 0000 0000) bin. At this time, the window width is 0.125 * 16 = 2.

When the inverse counter 170 operates in the form of bit parallelization, the inverse counter 170 subtracts the window width by 2 ⁿ and outputs it to the 1 counter 130. As shown in the embodiment illustrated in FIG. 2 (b), 1 counter 130 operates over 4 cycles of 2 ⁿ (n = 2), and the window width calculated and output by the inverse counter 170 is 8 If it is, the window width output by the inverse counter 170 in the first cycle is 8, and 4 is output in the next cycle.

The 1 counter 130 receives the window width output by the inverse counter 170 and the bit stream provided by the stochastic number generator 120, and counts the number of 1s in the window corresponding to the provided width.

In an embodiment in which the 1 counter 130 operates in the form of bit parallelization, the 1 counter 130 may receive a multi-bit input, and in a bit stream divided into a plurality of blocks, one bit included in each block The number of 1s included in each block can be counted by reading during the cycle (bit-parallelism). In one embodiment, when the bit stream provided by the stochastic number generator 120 is a total of 16 bits, and the window width is 4, one counter reads the bit stream by four bits in one cycle.

In one embodiment, the 1 counter 130 reads bits of data x equal to the width of the window to count the number of 1s, and does not process the rest. In one embodiment, if the data (x) provided to the 1 counter 130 is a binary number (1010 1011 1010 1010) bin, and the width of the window output by the inverse counter 170 is 4 bits, the 1 counter 130 is a bit The first 4 bits of the stream are read, and the number of 1s included is counted and the resulting decimal (2) dec is output to the accumulator 140.

As another example, if the data (x) provided to the 1 counter 130 is (1010 1011 1010 1010) bin, and the window width output by the inverse counter 170 is 8, the 1 counter 130 is the first 8 bits of the bit stream. Reads, counts the number of 1s included, and outputs the resulting decimal number (5) dec to the accumulator 140. The accumulator 140 accumulates and outputs the number provided by the 1 counter 130.

The result accumulated and output by the accumulator 140 corresponds to w × x, which is a product of an input (x) provided by the input providing unit 110 and a weight output by the weight providing unit 150. The XOR gate is multiplied and the accumulator 100 receives the sign bit of the input (x) provided by the input providing unit 110 and the sign bit of the quantized weight provided by the weight providing unit 150, and the sign of the multiplication result And outputs a sign bit to the accumulator 140. The accumulator 140 treats the multiplication result as a positive or negative number according to a sign bit and accumulates it.

3 (a) and 3 (b) are diagrams for explaining the operation of one counter 130 operating in bit parallelism. Equation 2 is an equation for explaining the operation of the 1 counter 130. Referring to FIG. 3, N = 16 in N bits where the input of 1 counter 130 is {x _N-1 , ..., x ₀ }, and 1 counter 130 is 16/2 ⁿ (n = 2) = 4 cycles of bit parallelism.

It is assumed that the width of the window is 8, as illustrated in FIG. 3 (a). Since the width of the window is greater than or equal to 2 ² = 4, the 1 counter 130 determines the number of “1s” included in the first block b1 and the second block b2 according to Equation (2). Count. In addition, since the 1 counter 130 operates by bit parallelization, the number of 1s in the first block b1 is counted, and then the number of 1s in the second block b2 is counted. In one embodiment, the window width provided by the inverse counter 170 in the first cycle may be 8, and may be 4 in the next cycle. The 1 counter 130 operating in bit parallelization counts the number of 1s included in the first block b1 in the first cycle, and the number of 1s included in the second block (b2) in the next cycle.

The first 3 bits of the first block b1 are fixed to x3, x2, and x3 as described above, and the last bit can be changed to x1 or x0. This can be expressed as (x3x2) bin + mux (x1, xo). For example, when x3 is 1, x2 is 0, and x1 is 1 in the first block (b1), the number of "1" included in the first block (b1) is (10) bin + (1) It can be calculated as bin = (11) bin = (3) dec. As described above, a signal for the multiplexer (MUX) to select x1 and x0 may be provided from a selected finite state machine (FSM).

In the second cycle, the 1 counter 130 counts the number of 1s included in the second block b2. Similar to the method of counting 1 in the first block (b1), (x3x2) bin + mux (x1, xo) can be calculated, and in the second block (b2), x0 is selected through the multiplexer. As in the above example, when x3 is 1, x2 is 0, and x0 is 0 in the second block (b1), the number of "1" included in the second block (b2) is (10) bin + ( 0) bin = (10) bin = (2) dec.

The method of counting the number of 1 when the width of the window is less than 2 ⁿ (n = 2) = 4 as illustrated in FIG. 3 (b) is as follows.

로 연산된다. 즉, 윈도우 너비(width)에 상응하는 비트 수를 x3가 나타내는 확률인 1/2에 곱하여 반올림(round)하면 윈도우 너비 내 x3가 포함된 개수를 얻을 수 있다. 마찬가지로, 윈도우 너비(width)인 2에 x2가 나타내는 확률인 1/4을 곱하여 반올림(round)하면 윈도우 너비 내 x2가 포함된 개수를 얻을 수 있다. 따라서, 이들 각각에 x3 값과 x2 값을 곱하면 윈도우 너비(width) 내의 1의 개수를 구할 수 있다.That is, in FIG. 3 (b), when x3 is 1 and x2 is 0, the number of 1s in the first 3 bits of the first block b1 is according to Equation (3).

Is calculated as That is, if the number of bits corresponding to the window width is multiplied by 1/2, which is the probability represented by x3, and rounded, the number of x3 in the window width can be obtained. Likewise, if you multiply the window width (2) by the probability that x2 represents 1/4, and round it up, you can get the number of x2 in the window width. Therefore, by multiplying each of these by the x3 value and the x2 value, the number of 1s in the window width can be obtained.

Another way to count the number of 1 is as follows.

(∧: bit-wise AND operation)

를 정수부와 소수부로 분할하여 고려한다. 정수부는 w를 i 번 우측으로 시프트 연산한 것과 동일하다. 즉, 정수부를 a라고 하면, a = w >> i 이다(>> : 시프트 연산). 나아가, 소수점 이하의 자리로 시프트되어 해당 수는 버려진다. 상기한 결과로부터 수학식 4의 첫 항이 도출된다. In Equation 3

Is considered by dividing into an integer part and a decimal part. The integer part is equivalent to the shift operation of w to the right of i times. That is, if the integer part is a, a = w >> i (>>: shift operation). Furthermore, the number is discarded because it is shifted to a decimal place. The first term of Equation 4 is derived from the above results.

In addition, in Equation 3, the value of the fractional part is rounded up. The rounding operation adds 1 to the integer part when the first digit of the decimal number is 1, and discards the following value when the first digit of the decimal number is 0. Therefore, it is possible to perform a rounding operation by performing a bit-wise operation and a value before shifting to a decimal point.

The number of 1 can be obtained by performing in this way.

As an example, suppose x = {1100} bin, and the window width is 2dec = 0010bin. Equation 3 is calculated as shown in Table 2 below.

Therefore, in the case of i = 1, i = 2 corresponding to the window width, adding the operation result of the integer part and the decimal part is 2 in decimal, which matches the number of 1 contained in x. Although the window width is 2, according to the hardware configuration, the above operation is performed for all cases where i = 1, i = 2, i = 3, and i = 4.

However, the window width is a number obtained by linearly converting the log, and there is only one 1 value in the binary value representing the window width. As in the above example, if x = {1100} bin, and the window width is 2dec = 0010bin, p is the position of 1 in the window width, p = 1. From this, Equation 4 is simply arranged as Equation 5 below.

Table 3 below shows the results of these calculations.

Although the window width is 2, according to the hardware configuration, the above operation is performed for all cases where i = 1, i = 2, i = 3, and i = 4.

If you add the result of the calculation of the integer and fractional parts, the decimal number is 2, which matches the number of 1s in x.

4 is a diagram showing an outline of one counter 130. 1 counter 130 illustrated in FIG. 4 is an example in which Equation 2 and Equation 5 are implemented in hardware. Referring to FIG. 4, the path illustrated by the bold solid line in FIG. 4 is a path for performing the operation of Equation (1). If the window width is greater than 2 ⁿ , the shifter 132 outputs the input signal without shifting, and the multiplexer (MUX) is input to the block among the input (x _Nn-1 , ..., x ₀ ). Select the bit to be included and output it.

In one embodiment, in the embodiment illustrated to count the number of 1 included in the first block (b1) in FIG. 3, (x3 x2 x3) bin passes through the shifter 132, but the bit is not shifted It is provided to the adder 134. The x1 bit of the first block b1 is selected by the multiplexer (MUX) and output to the adder 134. The adder 134 adds and outputs a value provided as an input.

에 상응한다. (x_N-1, ..., x_{N -n})에 포함된 비트들 중에서 다중화기(MUX)는 1의 위치(p)를 나타내는 신호(p)를 제공받고, x_N-p-1에 상응 하는 비트를 선택하여 덧셈기(134)에 출력한다. 덧셈기(134)는 두 경로를 통해 입력된 값들을 더하여 출력한다. The path illustrated by the dotted line in FIG. 4 is a path for calculating Equation 5. (x _N-1 , ..., x _Nn ) are input to the shifter 132 and are sequentially shifted according to a signal p indicating a position p of 1 in the window width to adder 134 Is provided on. The signal provided to the adder 134 according to the illustrated path

Corresponds to Among the bits included in (x _N-1 , ..., x _{N -n} ), the multiplexer (MUX) is provided with a signal p indicating the position p of ₁ , and corresponds to x _Np-1 . The bits are selected and output to the adder 134. The adder 134 adds and outputs the values input through the two paths.

Although not shown in FIG. 4, the shifter 132 is provided with a shift signal from a control unit not shown. The shifter 132 controls the number of bits in which the input signal is shifted according to the shift signal. The multiplexer (MUX) receives a selection signal from a control unit (not shown). The multiplexer (MUX) selects and outputs one of the input signals according to the selection signal.

5 is a diagram showing an outline of the accelerator 10 including the multiplier and accumulator 100 according to the present embodiment. Referring to FIG. 5, the accelerator 10 according to the present embodiment includes an input providing unit 210 providing input vectors, and a weight providing unit 250 providing log quantized weights.

The weight providing unit 210 outputs components (xa, xb, xc, and xd) of each input vector to the stochastic number generators 120a, 120b, 120c, and 120d. Stochastic number generators form a bit stream of the number of stochastics corresponding to the inputted components (xa, xb, xc, xd) and provide them to 1 counter 130a, 130b, 130c, 130d.

The log quantized weight value provided by the weight providing unit 250 is common to each component, is provided to the log-to-linear converter 160, and the inverse counter 170 forms a window width so that each 1 counter 130a, 130b, 130c, 130d). Each 1 counter 130a, 130b, 130c, 130d counts 1 within a provided window of each bit stream, outputs to each accumulator 140a, 140b, 140c, 140d, and each accumulator 140a, 140b , 140c, 140d) accumulate the provided values. Each XOR gate outputs a multiplication result to each accumulator 140a, 140b, 140c, 140d from the sign of the weight and the sign bit of each input vector component (xa, xb, xc, xd), and each accumulator 140a, 140b, 140c, 140d) adds a sign to the result provided by the 1 counter 130a, 130b, 130c, 130d and accumulates the result.

Log quantized SC multiplication accuracy analysis

FIG. 6 shows the multiplication error expressed as the mean square error of the stochastic computing accelerator. The uniformly distributed 16-bit prime including the sign bit was multiplied by the weight of Alexnet's first layer. In the case of quantized multiplication error, the actual multiplication output was calculated by multiplying floating-point weights. In comparison, linearly quantized weights or linear or logarithmic weights were multiplied by the actual output to calculate the multiplication error.

In a linear quantized stochastic computing accelerator, the quantized multiplication error is similar to the multiplication error, which means that the multiplication error in the linear quantization case is mainly due to the wrong multiplication itself. However, in the case of this embodiment, it indicates that the error mainly occurred in the log quantization error, and the multiplication error is lower than that in the linear quantization, and of the shifter based multiplication which is a fixed point multiplication for the multiplicand expressed in logarithm. Accuracy. In the case of the present embodiment quantized to log, the quantized multiplication error decreases at first, but it starts to saturate because the quantization error cannot be improved due to high data precision. For example, 0.7 is always quantized to 0.5 even if the data precision increases.

Small bitwidth quantization methods typically require retraining or fine tuning to restore the original network accuracy. A near accelerator including a stochastic computing MAC can achieve higher network accuracy with fine tuning. Network accuracy is more resilient to log quantization errors than multiplication errors, so the precision requirements for logarithmic quantized SCs can remain almost the same compared to modern SCs.

Evaluating the efficiency of area and power consumption

Accelerators consisting of 256 multipliers and accumulators were evaluated for all comparison cases. An accelerator based on a fixed-point binary design (Fixed), a shifter-based accelerator, a serial accelerator, and a bit parallel accelerator are compared technologies, and compared with the accelerator according to the present embodiment using log quantization. The accelerator according to the present embodiment includes a 1 counter in which a bottleneck is reduced in multipliers and accumulators using log quantized weights. The accelerator was implemented with Verilog RTL code and synthesized using Synopsys Design Compiler. The target standard cell library is TSMC 45nm and the target frequency is fixed at 1 GHz. Typical data precision is 16 bits. There is a bit parallel design and parallelized in 2 to ⁴ cycles for this evaluation (hardware precision is 4).

Figure 7 (a) is an accelerator area, Figure 5 (b) is a table showing the power consumption comparison. Referring to FIG. 7 (a), the fixed-point binary design (Fixed) includes a multiplier that occupies 80% or more of the total area. The area of the accumulator (Cntr / Acc., Counter of the serial SC) is similar for the comparison technology. Shifter-based accelerators are several times smaller than fixed-point binary designs.

The area of the serial accelerator is about 90% of the counter. The area of the bit parallel accelerator is nearly twice as large by the large 1s counter, but smaller than the area of the shift-based accelerator (Shift). In this embodiment, the area of 1 counter is about 1/5, and the total area depends on the area of the accumulator (Cnt / Acc). As a result, the area was reduced by 40% compared to the bit parallel accelerator.

Referring to FIG. 5B comparing power consumption, power consumption tends to be similar to area consumption. The power consumption of this embodiment is 84% and 39% lower, respectively, compared to the fixed point binary design (Fixed) based accelerator and the shift based accelerator. Compared to the bit parallel accelerator, the power consumption of this embodiment is 24% lower than the comparative technology, but 17% higher than the serial accelerator.

Efficiency of DNN application in log quantized stochastic computing

Since log quantization generates an error naturally compared to the original value, an explicitly acceptable level of neural network performance must be guaranteed. Image classification tests were performed with simulations that expanded the Caffe framework. Target Image Classification Only the convolutional layer, which performs most of the computation, has been accelerated. The target DNNs are LeNet and AlexNet for the 10 class data sets MNIST and CIFAR10. The large-scale AlexNet was tested with the 1000-class LSVRC-2010 ImageNet dataset. Floating-point learning weights were commonly fine-tuned 5,000 times.

FIG. 8 is a diagram showing the result of recognition test on a target neural network fine-tuned 5000 times, and FIG. 8 (a) is a 10-class image classification result, and FIG. 8 (b) is a 1000 class image classification result. Referring to FIG. 8 (a), the present embodiment implementing log quantization generally shows a better recognition rate for the technologies to be compared (5 bits and 9 bits are generally required for strict error limit conditions (12% loss)). Matter). This means that even if the quantization error is greater for the log quantization weight, the recognition rate can be restored by fine-tuning the weight. Furthermore, multiplication error in fine tuning is a bigger problem than quantization error. This phenomenon was further improved in the larger and more difficult neural network shown in Fig. 8 (b). The prior art SC starts learning at 10-bit precision and this embodiment of log quantization starts at 7-bit. Accuracy saturation starts at 8 bits, similar to a linear quantized binary fixed point. This embodiment is found to be more accurate than the linear quantization comparative example despite the smaller parameter size and larger quantization error.

However, since the prior art stochastic computing works the same as the present embodiment for the same input except for the sign bit processing, the sign size design of the prior art stochastic computing uses the fine-tuned weights to achieve the same accuracy as the present embodiment. Can be achieved. Therefore, although it cannot be said that this embodiment finally surpasses the stochastic computing of the prior art in terms of precision requirements, the present embodiment surpasses the fine tuning performance and does not lower the recognition rate compared to the prior art.

10: 100: multiplication and accumulator
110: input providing unit 120: stochastic number generator
130: 1 counter 132: shifter
134: adder 140: accumulator
150: weighting unit 160: log to linear converter
170: inverse counter

Claims (20)

An input providing unit providing input data;
A weight providing unit that receives a weight input and outputs a weight quantized into a log;
A log-to-linear converter that receives a quantized weight as the log and outputs a linearly converted weight;
An inverse counter for outputting a window width from the linearly converted weights;
1 counter for counting a value of 1 in the window width from the number of stochastics corresponding to the input data, and
A multiply-accumulator including an accumulator that accumulates and outputs the values output by the 1 counter. According to claim 1,
The multiplication accumulator,
The sign bit of the input data and
The multiplicative accumulator further comprises an exclusive logical (XOR, exclusive or) gate for computing the sign of the value output by the 1 counter from the sign bit in the weight quantized with the logarithm. According to claim 1,
The multiplication accumulator,
A multiplier accumulator further comprising a stochastic number generator that converts the input data into a stochastic number. According to claim 1,
The 1 counter
A multiplicative accumulator forming a number of stochastics corresponding to the input data. According to claim 1,
The 1 counter,
A multiplier accumulator that performs counting in a plurality of block units formed by dividing the stochastic number according to a predetermined number of bits. The method of claim 5,
When the width of the window is greater than or equal to the bit width of the block,
The 1 counter,
Equation

A multiplicative accumulator that counts the number of 1s in the block by computing. The method of claim 5,
When the width of the window is smaller than the bit width of the block,
The 1 counter,
Equation

A multiplier accumulator comprising a 1 counter that calculates and counts the number of 1s in the window width. The method of claim 5,
When the width of the window is smaller than the bit width of the block,
The 1 counter,
Equation

A multiplier accumulator including a 1 counter that calculates and counts the number of 1s in the window width. The method of claim 5,
When the width of the window is smaller than the bit width of the block,
The 1 counter,

A multiplier accumulator comprising a 1 counter that calculates and counts the number of 1s in the window width. The method of claim 5,
The 1 counter,
A shifter that receives a bit of the last bit from the first bit of the block and performs a right shift operation;
A multiplexer that selects and outputs one of the last previous bit and the last bit of the block as inputs from the first bit of the block;
An adder for adding the outputs of the shifter and the multiplexer; And
A multiplier accumulator comprising a counter that includes a control unit for selecting the number of bits shifted by the shifter and an input of the multiplexer. An input providing unit providing a plurality of input data vectors;
A weight providing unit that receives a weight input and outputs a weight quantized into a log;
A log-to-linear converter that receives a quantized weight as the log and outputs a linearly converted weight;
An inverse counter for outputting a window width from the linearly converted weights;
A plurality of 1 counters for counting a value of 1 in the window width from a number of stochastics corresponding to each of the input data vectors, and
A matrix-vector multiplier including the plurality of accumulators that accumulate and output the value output by the 1 counter. The method of claim 11,
The matrix vector multiplier (matrix-vector multiplier),
The sign bit of each of the input data vectors and
A matrix-vector multiplier further comprising the plurality of exclusive OR (XOR) gates that calculate the sign of the value output by the 1 counter from the sign bit in the log-quantized weight. The method of claim 11,
The multiplication accumulator,
A matrix-vector multiplier further comprising the plurality of stochastic number generators that convert each of the input data vectors to a corresponding stochastic number. The method of claim 11,
The 1 counter
A matrix-vector multiplier that forms a number of stochastics corresponding to the input data vector. The method of claim 11,
The 1 counter,
A matrix-vector multiplier that performs counting in a plurality of block units formed by dividing the number of stochastics according to a predetermined number of bits. The method of claim 15,
When the width of the window is greater than or equal to the bit width of the block,
The 1 counter,
Equation

A matrix-vector multiplier that calculates and counts the number of 1s in the block. The method of claim 15,
When the width of the window is smaller than the bit width of the block,
The 1 counter,
Equation

A matrix-vector multiplier including 1 counter for counting the number of 1s in the window width by calculating. The method of claim 15,
When the width of the window is smaller than the bit width of the block,
The 1 counter,
Equation

A matrix-vector multiplier including 1 counter for counting the number of 1s in the window width by calculating. The method of claim 15,
When the width of the window is smaller than the bit width of the block,
The 1 counter,

A matrix-vector multiplier including 1 counter for counting the number of 1s in the window width by calculating. The method of claim 15,
The 1 counter,
A shifter that receives a bit of the last bit from the first bit of the block and performs a right shift operation;
A multiplexer that selects and outputs one of the last previous bit and the last bit of the block as inputs from the first bit of the block;
An adder for adding the outputs of the shifter and the multiplexer; And
A matrix-vector multiplier comprising 1 counter including a control unit for selecting the number of bits shifted by the shifter and an input of the multiplexer.

Images