KR20200139071A - Method and apparatus for quantizing parameter in neural network - Google Patents

Abstract

A method and device for quantizing a parameter in a neural network determine a first quantization value by performing log quantization on a parameter of the neural network, determine a second quantization value by performing log quantization on an error between a first dequantization value obtained by dequanziation for the first quantization value and the parameter, and quantize the parameter to a value that the first quantization value and the second quantization value are grouped.

Description

A method and apparatus for quantizing a parameter in a neural network

The present disclosure relates to a method and apparatus for quantizing parameters in a neural network.

Neural network refers to a computational architecture that models a biological brain. With the recent development of neural network technology, researches for analyzing input data and extracting valid information using neural network devices in various types of electronic systems have been actively conducted.

Neural network devices require a large amount of computation on complex input data. In order for a neural network device to analyze an input in real time and extract information, a technology capable of efficiently processing a neural network operation is required. In particular, since a low-power, high-performance embedded system, such as a smartphone, has limited resources, a technology capable of minimizing loss of accuracy while reducing the amount of computation required to process complex input data is required.

It is to provide a method and apparatus for quantizing parameters in a neural network. Further, it is to provide a computer-readable recording medium in which a program for executing the method on a computer is recorded.

The technical problem to be solved is not limited to the technical problems as described above, and other technical problems can be inferred.

According to an aspect, a method of quantizing a parameter in a neural network includes first quantization by performing log quantization on the parameter including at least one of input activation values and weight values processed in a layer of the neural network. Determining a value; Comparing an error between the parameters and a threshold value of a first inverse quantization value obtained by dequantization of the first quantization value; Determining a second quantization value by performing log quantization on the error when the error is greater than the threshold value as a result of the comparison; And quantizing the parameter into a grouped value of the first quantization value and the second quantization value.

In addition, the determining of the first quantization value may include determining the first quantization value by performing log quantization on a value corresponding to a quantization level closest to the parameter among the plurality of quantization levels, and In determining the 2 quantization value, the second quantization value is determined by performing log quantization on a value corresponding to the quantization level closest to the error among the plurality of quantization levels.

Also, the second quantization value may be expressed by the same number of bits as the number of bits representing the first quantization value.

Also, the threshold value may be determined based on a trade-off relationship between a recognition rate of the neural network and a size of data according to quantization of the parameter.

In addition, the quantizing step may further include adding a tag bit to each of the first quantization value and the second quantization value.

In addition, the step of adding may include a first tag bit indicating that the second quantization value is present in the first quantization value before or after the first bit of the bits indicating the first quantization value. In addition, a second tag bit indicating that there is no consecutive quantization value in the second quantization value may be added before the first bit or after the last bit among the bits indicating the second quantization value.

In addition, in the quantizing step, the first quantization value and the second quantization value, in any one of the front of the first bit of the bits constituting the first quantization value and the rear of the last bit of the bits representing the second quantization value, and It may further include the step of adding a code value indicating that the second quantization value is a continuous value.

In addition, a method of quantizing a parameter including at least one of input activation values and weight values processed in a layer of a neural network may include inverse quantizing the grouped values; And performing a convolution operation between a value obtained by inverse quantization of the grouped value and the input activation values.

The inverse quantization of the grouped values may include calculating a first inverse quantization value that is a value obtained by inverse quantization of the first quantization value and a second inverse quantization value that is a value obtained by inverse quantization of the second quantization value; And obtaining an inverse quantization value from the grouped value by adding the first inverse quantization value and the second inverse quantization value.

According to another aspect, an apparatus for implementing a neural network, comprising: a memory; And a processor that quantizes a parameter including at least one of input activation values and weight values processed in a layer of the neural network, wherein the processor performs log quantization on the parameter to determine the first quantization value. Determine and compare an error and a threshold value between a first inverse quantization value obtained by dequantization of the first quantization value and the parameter, and if the error is greater than the threshold value, A second quantization value is determined by performing logarithmic quantization, and the parameter is quantized into a value obtained by grouping the first quantization value and the second quantization value.

In addition, the processor determines the first quantization value by performing log quantization on a value corresponding to a quantization level closest to the parameter among the plurality of quantization levels, and the error among the plurality of quantization levels The second quantization value may be determined by performing log quantization on the value corresponding to the quantization level closest to.

Also, the second quantization value may be expressed by the same number of bits as the number of bits representing the first quantization value.

Also, the threshold value may be determined based on a trade-off relationship between a recognition rate of the neural network and a size of data according to quantization of the parameter.

In addition, the processor may add a tag bit to each of the first quantization value and the second quantization value.

In addition, the processor adds a first tag bit indicating that the second quantization value is contiguous to the first quantization value, before or after the first bit of the bits indicating the first quantization value, and , A second tag bit indicating that there is no consecutive quantization value in the second quantization value may be added before the first bit or after the last bit among bits indicating the second quantization value.

In addition, the processor may include the first quantization value and the first quantization value in any one of bits before a first bit among bits representing the first quantization value and after a last bit among bits constituting the second quantization value. 2 A code value indicating that the quantization value is a continuous value can be added.

In addition, the processor may inverse quantize the grouped values and perform a convolution operation between a value obtained by inverse quantization of the grouped values and the input activation values.

In addition, the processor calculates each of a first inverse quantization value that is a value obtained by inverse quantization of the first quantization value and a second inverse quantization value that is a value obtained by inverse quantization of the second quantization value, and the first inverse quantization value and the An inverse quantization value may be obtained from the grouped value by adding a second inverse quantization value.

According to another aspect, the computer-readable recording medium may include a computer-readable recording medium recording a program for executing the above-described method on a computer.

1 is a diagram for describing an example of a neural network architecture.
2 is a diagram for describing an example of an operation performed in a neural network.
3 is a diagram showing an example of a linear quantization method and a log quantization method.
4 is a diagram showing an example of a quantization error for each of a linear quantization method and a log quantization method.
5 is a flowchart illustrating an example of a method of quantizing parameters in a neural network.
6 shows an example in which a neural network device determines a first quantization value.
7 shows an example in which a neural network device determines whether a difference between a first inverse quantization value and a parameter is greater than a threshold value.
8 is a diagram illustrating an example in which a neural network device determines a second quantization value.
FIG. 9 is a diagram illustrating an example in which a first quantization value and a second quantization value are quantized values of one parameter using a tag bit.
10 is a diagram illustrating an example in which a first quantization value and a second quantization value are quantized values of one parameter using a code value.
11 is a flowchart illustrating an example of a method for quantizing a parameter by a neural network device.
12 is a flowchart illustrating an example of a method of inverse quantizing a parameter by a neural network device.
13 is a diagram comparing accuracy according to a method of quantizing parameters in a neural network.
14 is a diagram illustrating an example of a computing device that performs a multiplication and accumulation (MAC) operation.
15 is a diagram illustrating another example of a computing device that performs a multiplication and accumulation (MAC) operation.
16 is a block diagram illustrating an electronic system according to an exemplary embodiment.
17 is a block diagram illustrating an example of a configuration of a neural network device.

As for the terms used in the embodiments, general terms that are currently widely used as possible are selected, but this may vary according to the intention or precedent of a technician working in the art, and the emergence of new technologies. In addition, in certain cases, there are terms that are arbitrarily selected, and in this case, the meaning will be described in detail in the description of the corresponding embodiment. Accordingly, terms used in the specification should be defined based on the meaning of the term and the overall contents of the present embodiments, not a simple name of the term.

Throughout the specification, when a part is said to be connected to another part, this includes not only a case in which it is directly connected, but also a case in which it is electrically connected with another component interposed therebetween. In addition, when a certain part includes a certain component, it means that other components may be further included, rather than excluding other components unless specifically stated to the contrary.

Terms such as “consisting of” or “comprising” described in the specification should not be construed as necessarily including all of the various elements or various steps described in the specification, and some of the elements or some steps are included It should be construed that it may not be, or may further include additional components or steps.

Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings. However, the embodiments may be implemented in various different forms and are not limited to the examples described below.

1 is a diagram for describing an example of a neural network architecture.

The neural network 1 illustrated in FIG. 1 may be an example of a deep neural network (hereinafter referred to as “DNN”) architecture. DNN may correspond to Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Deep Belief Networks, Restricted Boltzman Machines, etc., but is not limited thereto. The neural network 1 may be a DNN including an input layer (Layer 1), four hidden layers (Layer 2, Layer 3, Layer 4, and Layer 5), and an output layer (Layer 6). For example, when the neural network 1 represents a convolutional neural network (CNN), Layers 1 to 6 may correspond to some layers of the convolutional neural network, which is a convolutional layer, It may correspond to a pooling layer, a fully connected layer, and the like.

Each of the layers included in the neural network 1 is a plurality of artificial nodes, known as "neurons", "processing elements (PE)", "units" or similar terms. node). For example, as shown in FIG. 1, Layer 1 may include 5 nodes, and Layer 2 may include 7 nodes. However, this is only an example, and each of the layers included in the neural network 1 may include various numbers of nodes.

Nodes included in each of the layers included in the neural network 1 may be connected to each other to exchange data. For example, one node may receive data from other nodes and perform an operation, and output the operation result to other nodes.

Meanwhile, when the neural network 1 is implemented with a DNN architecture, it may include many layers capable of processing valid information. Therefore, the neural network 1 can process more complex data sets than a neural network including a single layer. Meanwhile, although FIG. 1 shows that the neural network 1 includes 6 layers, this is only an example, and the neural network 1 may include fewer or more layers. That is, the neural network 1 may include layers of various structures different from those shown in FIG. 1.

2 is a diagram for describing an example of an operation performed in a neural network.

및

)를 기초로 연산을 수행하고, 수행 결과를 기초로 출력 데이터(예를 들어,

및

)를 생성할 수 있다.2, the neural network 2 has a structure including an input layer, hidden layers, and an output layer, and received input data (for example,

And

) To perform an operation, and output data (for example,

And

) Can be created.

As described above, the neural network 2 may be a DNN or an n-layer neural network including two or more hidden layers. For example, as shown in FIG. 2, the neural network 2 may be a DNN including an input layer (Layer 1), two hidden layers (Layer 2 and Layer 3), and an output layer (Layer 4). have. When the neural network 2 is implemented with a DNN architecture, since it includes more layers capable of processing valid information, the neural network 2 can process more complex data sets than a neural network having a single layer. Meanwhile, the neural network 2 is shown to include four layers, but this is only an example, and the neural network 2 may include fewer or more layers, or fewer or more channels. That is, the neural network 2 may include layers of various structures different from those shown in FIG. 2.

Each of the layers included in the neural network 2 may include a plurality of channels. A channel may correspond to a plurality of artificial nodes, known as neurons, processing elements (PEs), units, or similar terms. For example, as shown in FIG. 2, Layer 1 may include two channels (nodes), and each of Layer 2 and Layer 3 may include three channels. However, this is only an example, and each of the layers included in the neural network 2 may include a variety of channels (nodes).

Channels included in each of the layers of the neural network 2 are connected to each other to process data. For example, one channel may receive data from other channels and perform an operation, and the operation result may be output to other channels.

Each of the input and output of each of the channels may be referred to as input activation and output activation. That is, activation may be an output of one channel and a parameter corresponding to an input of channels included in the next layer. Meanwhile, each of the channels may determine its own activation based on activations, weight and bias received from channels included in the previous layer. The weight is a parameter used to calculate the output activation in each channel, and may be a value assigned to a connection relationship between channels.

는 액티베이션 함수(activation function)이고,

는 (i-1) 번째 레이어에 포함된 k 번째 채널로부터 i 번째 레이어에 포함된 j번째 채널로의 웨이트며,

는 i 번째 레이어에 포함된 j 번째 채널의 바이어스(bias)이고,

는 i 번째 레이어의 j 번째 채널의 액티베이션이라고 할 때, 액티베이션

는 다음과 같은 수학식 1을 이용하여 계산될 수 있다.Each of the channels may be processed by a computational unit or a processing element that receives an input and outputs an output activation, and the input-output of each of the channels may be mapped. For example,

Is the activation function,

Is the weight from the k-th channel included in the (i-1)-th layer to the j-th channel included in the i-th layer,

Is the bias of the j-th channel included in the i-th layer,

Is the activation of the j-th channel of the i-th layer,

Can be calculated using Equation 1 as follows.

로 표현될 수 있다. 또한,

은 수학식 1에 따라

의 값을 가질 수 있다. 다만, 앞서 설명한 수학식 1은 뉴럴 네트워크(2)에서 데이터를 처리하기 위해 이용되는 액티베이션 및 웨이트 및 바이어스를 설명하기 위한 예시일 뿐, 이에 제한되지 않는다. 액티베이션은 이전 레이어로부터 수신된 액티베이션들의 가중치 합(weighted sum)을 sigmoid 함수나 Rectified Linear Unit (ReLU) 함수 등의 액티베이션 함수에 통과시킴으로써 획득된 값일 수 있다.As shown in FIG. 2, activation of the first channel (CH 1) of the second layer (Layer 2) is

It can be expressed as Also,

Is according to Equation 1

It can have a value of However, Equation 1 described above is only an example for explaining activation, weight, and bias used to process data in the neural network 2, and is not limited thereto. The activation may be a value obtained by passing a weighted sum of activations received from a previous layer through an activation function such as a sigmoid function or a Rectified Linear Unit (ReLU) function.

In general, parameters of a floating-point format and a fixed-point format may be used as inputs to the layers. In addition, parameters of a floating-point format and a fixed-point format may be used as weight and bias values.

As described above, in the neural network 2, numerous data sets are exchanged between a plurality of interconnected channels, and a number of calculation processes are performed while passing through layers. Accordingly, there is a need for a technique capable of minimizing loss of accuracy while reducing the amount of computation required to process complex input data.

3 is a diagram showing an example of a linear quantization method and a log quantization method.

In order to reduce the amount of computation generated in the neural network, studies on various quantization methods of parameters included in the neural network have been conducted. For example, by quantizing the weight parameters into bits having a length smaller than the original bit length, it is possible to reduce the amount of computation required for processing the weight parameters even though the accuracy is slightly reduced. Such quantization methods include a linear quantization method and a log quantization method.

The linear quantization method is a quantization method in which quantization levels at regular intervals are allocated to parameters of a neural network to be quantized. On the other hand, since the logarithm quantization method allocates quantization levels based on logarithm data representation, a relatively small quantization level is allocated to parameters near large values, and relatively large quantization levels are assigned to parameters near zero. It is a quantization method to allocate.

For example, referring to FIG. 3, a real value x of a parameter existing in the interval -1 to 1 may be quantized to a quantized integer value q. q may be an integer value between -M and M-1, and M may be 2N-1, for example.

According to the linear quantization method, x in the interval -1 to -(1-1/M) can be quantized to an integer value -M, and in the interval -(1-1/M) to -(1-2/M) x can be quantized to an integer value -(M-1). Likewise, x in the interval 1-1/M to 1 may be quantized to an integer value M-1.

On the other hand, according to the logarithmic quantization method, x in the interval -2-M to -2-(M-1) can be quantized to an integer value -M, and -2-(M-1) to --2-(M -2) x in the interval can be quantized to an integer value -(M-1). Likewise, x in the interval 2-(M-1) to 2-M may be quantized to an integer value M-1.

4 is a diagram illustrating an example of a quantization error for each of a linear quantization method and a log quantization method.

The quantization error may represent a difference between an actual value x of a parameter and a quantized integer value q. 4 is a graph showing a quantization error value according to an actual value x of a parameter.

When the actual value x of the parameter is a value approaching 0, the quantization error value according to the log quantization method may be less than or equal to the quantization error value according to the linear quantization method. However, as the actual value x of the parameter approaches 1, the quantization error value by the log quantization method may be larger than the quantization error value by the linear quantization method. Meanwhile, as the quantization error value increases, the recognition rate of the neural network may decrease. Therefore, a quantization method is required to solve the phenomenon that the quantization error value increases when the actual value x of the parameter approaches 1.

Accordingly, in the conventional log quantization method, a parameter is expressed as one quantization value, whereas the present embodiment proposes a log quantization method that expresses two or more quantization values when the quantization error value is greater than or equal to a threshold value. In addition, the present embodiment may aim to generate a neural network having processing functions such as quantizing and storing parameters in a neural network using the proposed quantization method, and performing an operation by inverse quantizing the parameters. Hereinafter, it will be described in detail with reference to FIGS. 5 to 17.

5 is a flowchart illustrating an example of a method of quantizing parameters in a neural network.

Referring to FIG. 5, a method of quantizing a parameter in a neural network includes steps processed in a time series in the neural network device shown in FIG. 17. Accordingly, it can be seen that even though the contents are omitted below, the contents to be described later with respect to the neural network device shown in FIG. 17 are also applied to the method of FIG. 5.

In step 510, the neural network device may determine the first quantization value by performing log quantization on the parameter.

The parameter may include at least one of activation values processed by a layer of a neural network and weight values processed by a layer, but is not limited thereto.

6 shows an example in which a neural network device determines a first quantization value.

Referring to FIG. 6, the neural network device determines a first quantization value based on log quantization having a base of 2, but is not limited thereto, and the base may correspond to various values other than 2. The neural network apparatus may determine the first quantization value by performing log quantization on a value corresponding to a quantization level closest to a parameter among a plurality of quantization levels.

For example, if the actual value of the parameter is 0.5 (=2-1), the neural network device expresses 1 corresponding to the absolute value of -1 corresponding to the exponent of 2 in binary and determines this as the first quantization value. I can. The neural network device may represent the first quantization value as 5 bits, and the first quantization value is 00001. However, the present invention is not limited thereto, and the first quantization value may be expressed by various number of bits.

Alternatively, if the actual value of the parameter is -0.25 (=-2-2), the neural network device expresses 2 corresponding to the absolute value of -2 corresponding to the exponent of 2 in binary and determines this as the first quantization value. I can. In this case, since the actual value of the parameter is a negative number, the neural network device may determine the first quantization value to be 11110 using a two's complement method.

Alternatively, when the actual value of the parameter is 0.4375, the neural network device may determine 00001 corresponding to 0.5 (=2-1) close to 0.4375 as the first quantization value. However, the present invention is not limited thereto, and the first quantization value may be variously determined according to an interval between quantization levels allocated to a parameter of a neural network. For example, a quantization level corresponding to 0.5 (=2-1) may not be included in a plurality of quantization levels allocated to a parameter of a neural network. In this case, 00010 may be determined as the first quantization value by performing log quantization of 0.25 (=2-2), which is a value corresponding to a quantization level closest to 0.4375 among a plurality of allocated quantization levels.

Returning to FIG. 5 again, in step 520, the neural network apparatus may compare the first inverse quantization value obtained by dequantization of the first quantization value and the error and the threshold value between the parameters.

The neural network device may obtain a first inverse quantization value by performing dequantization (inverse quantization) on the first quantization value according to Equation 2 below.

는 제 1 양자화 값을 역양자화한 제 1 역양자화 값에 해당할 수 있다. In Equation 2, q may correspond to a first quantization value,

May correspond to a first inverse quantization value obtained by inverse quantization of the first quantization value.

When the error between the first inverse quantization value and the parameter is smaller than the threshold value, the parameter may be quantized to the first quantization value. However, when the error is greater than the threshold value, a problem may occur that the recognition rate of the neural network is deteriorated, and thus the parameter may be expressed as two or more quantization values including the first quantization value.

7 shows an example in which a neural network device determines whether a difference between a first inverse quantization value and a parameter is greater than a threshold value.

As described above in FIG. 6, when the actual value of the parameter is 0.5 (=2-1), the first quantization value is 00001. Based on Equation 1, a first inverse quantization value of 0.5 may be obtained by inverse quantization of 00001, which is a first quantization value. The difference between 0.5, which is the actual value of the parameter, and 0.5, which is the first inverse quantization value, is 0. Accordingly, since the difference between the actual value of the parameter and the first inverse quantization value is less than 1/128, which is the threshold value, the parameter can be finally quantized to 00001.

Similarly, even when the actual value of the parameter is -0.25 (=2-2), the difference between the actual value of the parameter and the first inverse quantization value obtained by inverse quantization of 11110 is 0, and thus the parameter can be finally quantized to 11110.

이다. 따라서, 파라미터는 제 1 양자화 값인 00001을 포함하는 두 개 이상의 양자화 값으로 표현될 수 있으며, 이는 도 8에서 후술하도록 한다. Meanwhile, when the actual value of the parameter is 0.4375, the first quantization value is 00001. The first inverse quantization value obtained by inverse quantization of 00001 is 0.5, so the difference between the actual value of the parameter 0.4375 and the first inverse quantization value 0.5 is

to be. Accordingly, the parameter may be expressed as two or more quantization values including 00001 as the first quantization value, which will be described later in FIG. 8.

Meanwhile, in FIG. 7, the threshold value is set to 1/128, but is not limited thereto, and the threshold value may correspond to various values. For example, the threshold may be determined based on a recognition rate of a neural network and a size of data according to quantization of a parameter. As the threshold value is smaller, the actual value of the parameter can be quantized with high accuracy, so the recognition rate of the neural network increases, but the size of the data may increase as the parameter is expressed as multiple quantization values. Accordingly, the threshold value may be determined in consideration of a trade-off relationship between the recognition rate of the neural network and the size of data corresponding to the quantization value.

Referring back to FIG. 5, in step 530, when the comparison result error is greater than the threshold value, the neural network apparatus may determine the second quantization value by performing log quantization on the error.

8 is a diagram illustrating an example in which a neural network device determines a second quantization value.

Similar to when determining the first quantization value, the neural network apparatus may determine the second quantization value by performing log quantization on a value corresponding to a quantization level closest to an error among a plurality of quantization levels. In this case, the second quantization value may be expressed by the same number of bits as the number of bits representing the first quantization value. For example, when the first quantization value is represented by 5 bits, the neural network device may also represent the second quantization value by 5 bits. However, the present invention is not limited thereto, and the first quantization value and the second quantization value may be represented by different number of bits.

As described above in FIG. 7, an error between 0.4375, which is an actual value of the parameter, and 0.5, which is a first inverse quantization value obtained by inverse quantization of the first quantization value 00001, is -0.0625. The neural network device may perform log quantization on -0.0625 (-2-4) corresponding to the error. In this case, since the error value is a negative number, the neural network apparatus may determine the second quantization value as 11110 using a two's complement method.

Returning to FIG. 5 again, in step 540, the neural network apparatus may quantize the parameter into a value obtained by grouping the first quantization value and the second quantization value.

For example, the grouped value may correspond to a value in which a second quantization value is consecutively positioned after the first quantization value to form one group. Referring to FIG. 8, when the actual value of the parameter is 0.4375, the neural network device may quantize the parameter into a grouped value of 00001 corresponding to a first quantization value and 11110 corresponding to a second quantization value.

Meanwhile, an additional coding process may be performed to indicate that the first quantization value and the second quantization value are quantized values of one parameter, which will be described later in FIGS. 9 and 10.

FIG. 9 is a diagram illustrating an example in which a first quantization value and a second quantization value are quantized values of one parameter using a tag bit.

The neural network apparatus may add a tag bit to each of the first quantization value and the second quantization value to indicate that the first quantization value and the second quantization value are quantized values of one parameter.

For example, the neural network device may add a first tag bit indicating that there is a second quantization value consecutive to the first quantization value, before the first bit or after the last bit among the bits indicating the first quantization value. have. In addition, a second tag bit indicating that there is no consecutive quantization value to the second quantization value may be added before the first bit or after the last bit among bits indicating the second quantization value. When the first tag bit is 1, the second tag bit may be 0, but is not limited thereto. When the first tag bit is 0, the second tag bit may be 1.

Referring to FIG. 9, any one of the first tag bit and the second tag bit may be added before the first bit of bits representing each of quantization values. In this case, the first tag bit may correspond to 1, and the second tag bit may correspond to 0. Since 0 corresponding to the second tag bit is added in front of the first bit among the bits representing each of the quantization values q[n-1], q[n]2, and q[n+1], q[n-1] ], q[n]2, and q[n+1] respectively have no consecutive quantization values. Meanwhile, since 1 corresponding to the first tag bit is added in front of the first bit among the bits representing q[n]1, it can be seen that q[n]1 has a continuous quantization value. Accordingly, q[n]1 and q[n]2 are consecutively positioned to correspond to grouped values, and the grouped values may represent quantized values of one parameter.

As an example, when the first quantization value is 00001 and the second quantization value is 11100, 1 corresponding to the first tag bit is added to the front of the first bit among bits representing 00001, and the first of the bits representing 11100 0 corresponding to the second tag bit may be added in front of the th bit. Through this, the first quantization value 00001 and the second quantization value 11100 may be grouped to indicate that one parameter is quantized.

10 is a diagram illustrating an example in which a first quantization value and a second quantization value are quantized values of one parameter using a code value.

The neural network apparatus may add a specific code value to indicate that the first quantization value and the second quantization value are quantized values of one parameter. The code value may be composed of the same number of bits as the first quantization value or the second quantization value, but is not limited thereto.

For example, the neural network apparatus may include a first quantization value and the second quantization value in any one of bits before a first bit among bits representing a first quantization value and after a last bit among bits representing a second quantization value. A code value indicating that the value is a continuous value can be added.

Referring to FIG. 10, a code value may be added before the first bit among bits representing a quantization value. Since a code value is not added in front of the first bit among the bits representing each of the quantization values q[n-1], q[n]2, and q[n+1], q[n-1], q[n It can be seen that there are no consecutive quantization values for each of ]2 and q[n+1]. Meanwhile, since a code value is added in front of the first bit among the bits representing q[n]1, it can be seen that q[n]1 has a continuous quantization value. Accordingly, q[n]1 and q[n]2 are consecutively positioned to correspond to grouped values, and the grouped values may represent quantized values of one parameter.

As an example, when the first quantization value is 00001 and the second quantization value is 11100, a code value (-M) may be added in front of the first bit among bits representing 00001. Through this, the first quantization value 00001 and the second quantization value 11100 may be grouped to indicate that one parameter is quantized.

11 is a flowchart illustrating an example of a method for quantizing a parameter by a neural network device.

In step 1110, the neural network device may set k=1, δ=x, and q1=0 as initial conditions. x may correspond to an actual value of a parameter to be quantized by the neural network device.

In step 1120, the neural network device may determine whether a value corresponding to k is greater than L. L may correspond to the number of quantization values that can be grouped to a maximum when a parameter is quantized into a value in which a plurality of quantization values are grouped. If the value corresponding to k is not greater than L, the process proceeds to step 1130. If the value corresponding to k is greater than L, the process proceeds to step 1180.

In step 1130, the neural network device may determine whether a value corresponding to δ is 0. If the value corresponding to δ is 0, the process proceeds to step 1140. If the value corresponding to δ is not 0, the process proceeds to step 1180.

In step 1140, the neural network device may determine the quantization value qk by performing log quantization. qk may be determined by Equation 3 below.

은 δ의 부호에 따라 1 또는 -1로 결정될 수 있고,

는 δ의 절대값에 해당하며,

은

에 해당하는 값을 소수점 첫째 자리에서 반올림한 값에 해당할 수 있다.In Equation 3 above

May be determined as 1 or -1 depending on the sign of δ,

Corresponds to the absolute value of δ,

silver

It may correspond to a value that is rounded to the first decimal place.

In step 1150, the neural network apparatus may calculate an error between an actual value of a parameter and a value obtained by inverse quantization of a grouped value of the quantization values q1 to qk. The error may be determined by Equation 4 below.

는 양자화 값 q1 내지 qk가 그룹핑된 값을 역양자화한 값에 해당할 수 있고,

은

의 절대값에 해당할 수 있다. 한편, 복수의 양자화 값들이 그룹핑된 값을 역양자화하는 것에 관해서는 도 12에서 후술한다. In Equation 4 above

May correspond to a value obtained by inverse quantization of a grouped value of quantization values q1 to qk,

silver

It can correspond to the absolute value of Meanwhile, inverse quantization of a grouped value of a plurality of quantization values will be described later in FIG. 12.

In step 1160, the neural network device may determine whether a value corresponding to δ is greater than a threshold value O. If the value corresponding to δ is greater than the threshold value O, the process proceeds to step 1170. If the value corresponding to δ is not greater than the threshold value O, the process proceeds to step 1180.

In step 1170, the neural network device may increase the value corresponding to k by 1.

In step 1180, the neural network apparatus may finally quantize the parameter to a value obtained by grouping the quantization values q1 to qk.

12 is a flowchart illustrating an example of a method of inverse quantizing a parameter by a neural network device.

으로 설정할 수 있다. 이 때, 역양자화의 대상이 되는 값은 양자화 값 q1 내지 qn이 그룹핑된 값일 수 있다.

는 그룹핑된 값을 역양자화한 값에 해당할 수 있다. In step 1210, the neural network device k=1 as an initial condition,

Can be set to In this case, a value to be subjected to inverse quantization may be a value obtained by grouping quantization values q1 to qn.

May correspond to a value obtained by inverse quantization of the grouped values.

In step 1220, the neural network device may determine whether a value corresponding to qk is 0. If the value corresponding to qk is not 0, the process proceeds to step 1230. If the value corresponding to qk is 0, the process proceeds to step 1240.

를 결정할 수 있다.

는 하기 수학식 5에 의해 결정될 수 있다.In step 1230, the neural network device performs inverse quantization to determine the inverse quantization value.

Can be determined.

May be determined by Equation 5 below.

은 δ의 부호에 따라 1 또는 -1로 결정될 수 있다. 양자화 값 q1 내지 qn이 그룹핑된 값을 역양자화한 값

는 양자화 값 q1 내지 qn 각각을 역양자화한 값을 누적함으로써 결정될 수 있다.In Equation 5 above

May be determined as 1 or -1 depending on the sign of δ. Inverse quantization of the grouped values of quantization values q1 to qn

May be determined by accumulating values obtained by inverse quantization of each of the quantization values q1 to qn.

In step 1240, the neural network device may determine whether a value corresponding to k is n. If the value corresponding to k is n, the process proceeds to step 1260. If the value corresponding to k is not n, the process proceeds to step 1250.

In step 1250, the neural network device may increase the value corresponding to k by 1.

를 출력할 수 있다.

는 양자화 값 q1 내지 qn 각각을 역양자화한 값을 누적한 값에 해당할 수 있다.In step 1260, the neural network device is a value obtained by inverse quantizing the grouped values of quantization values q1 to qn.

Can be printed.

May correspond to an accumulated value obtained by inverse quantization of each of the quantization values q1 to qn.

13 is a diagram comparing the accuracy of neural networks according to a method of quantizing parameters.

Models of neural networks for calculating the accuracy of image classification may include AlexNet, SqueezeNet, and VGG-S. Referring to FIG. 13, when a linear quantization method, a log quantization method, and a log quantization method expressed by two or more quantization values are applied when a quantization error value is greater than or equal to a threshold value according to the present embodiment, classification accuracy in a neural network (%) is indicated. On the other hand, float 32b indicates a case where the parameter is not quantized.

Referring to FIG. 13, it can be seen that when the log quantization method according to the present embodiment is applied compared to the conventional log quantization method, accuracy is improved by about 2% to 4%.

14 is a diagram illustrating an example of a computing device that performs a multiplication and accumulation (MAC) operation.

The neural network device may quantize weight values processed in a layer of a neural network and store them in a memory. Thereafter, the neural network apparatus may inverse quantize the quantized weight values to perform a convolution operation with input activation values.

In this case, when the weight value is quantized into a grouped value of the first quantization value and the second quantization value, the neural network device dequantizes the grouped value, and convolutions between the dequantized value of the grouped value and the input activation value. Can perform operations. Meanwhile, the neural network device calculates each of a first inverse quantization value that is a value obtained by inverse quantization of the first quantization value and a second inverse quantization value that is a value obtained by inverse quantization of the second quantization value Grouped values can be inverse quantized by adding values.

Accordingly, the first inverse quantization value, which is a value obtained by inverse quantization of the first quantization value, and the second inverse quantization value, which is a value obtained by inverse quantization of the second quantization value, must perform the same input activation value and operation.

14 is a computing device that performs a MAC operation when weight values are quantized using tag bits, where q represents a quantized value of weight values and X represents an input activation value. When the tag bit is 1, the computing device performing the MAC operation may use this as a multiplexer (MUX) selection signal so that the input activation value X of the previous cycle is continuously used for calculation. In this case, when the tag bit added to the quantization value is 1, it may indicate that there is another quantization value continuously positioned in the corresponding quantization value. When the tag bit is 0, since there is no quantization value positioned continuously, a new input activation value X different from the input activation value X of the previous cycle may be input to perform an operation.

15 is a diagram illustrating another example of a computing device that performs a multiplication and accumulation (MAC) operation.

FIG. 15 is a computing device that performs MAC operation when weight values are quantized using code values. Likewise, q represents a quantized value of weight values and X represents an input activation value. When a code value is input, the computing device performing MAC operation may use it as a multiplexer (MUX) selection signal. Accordingly, for the first quantization value and the second quantization value continuously input after the code value, the same input activation value X can be continuously used for calculation. Meanwhile, the calculation result between the code value and the input activation value X may not be accumulated in the result value Y.

16 is a block diagram illustrating an electronic system according to an exemplary embodiment.

Referring to FIG. 16, the electronic system 1600 extracts valid information by analyzing input data in real time based on a neural network, and makes a situation determination based on the extracted information, or an electronic system on which the electronic system 1600 is mounted. You can control the configuration of the device. For example, the electronic system 1600 is a robot device such as a drone, an advanced driver assistance system (ADAS), a smart TV, a smart phone, a medical device, a mobile device, an image display device, a measurement device, an IoT device. It may be applied to, etc., and may be mounted on at least one of various types of electronic devices.

The electronic system 1600 may include a CPU 1610, a RAM 1620, a neural network device 1630, a memory 1640, a sensor module 1650, and a communication module 1660. The electronic system 1600 may further include an input/output module, a security module, and a power control device. Some of the hardware components of the electronic system 1600 may be mounted on at least one semiconductor chip. The neural network device 1630 may be the above-described neural network dedicated hardware accelerator itself or a device including the same.

The CPU 1610 controls the overall operation of the electronic system 1600. The CPU 1610 may include one processor core (Single Core), or may include a plurality of processor cores (Multi-Core). The CPU 1610 may process or execute programs and/or data stored in the memory 1640. In an embodiment, the CPU 1610 may control functions of the neural network device 1630 by executing programs stored in the memory 1640. The CPU 1610 may be implemented as a CPU, GPU, or AP.

The RAM 1620 may temporarily store programs, data, or instructions. For example, programs and/or data stored in the memory 1640 may be temporarily stored in the RAM 1620 according to the control of the CPU 1610 or a boot code. The RAM 1620 may be implemented as a memory such as dynamic RAM (DRAM) or static RAM (SRAM).

The neural network apparatus 1630 may perform an operation of a neural network based on the received input data, and may generate an information signal based on a result of the execution. The neural network may include, but is not limited to, Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Deep Belief Networks, Restricted Boltzman Machines, and the like. The neural network device 1630 is hardware that performs processing using a neural network quantized to the fixed-point type described above, and may correspond to the aforementioned hardware accelerator dedicated to a neural network.

The information signal may include one of various types of recognition signals such as a voice recognition signal, an object recognition signal, an image recognition signal, and a biometric information recognition signal. For example, the neural network apparatus 1630 may receive frame data included in a video stream as input data, and may generate a recognition signal for an object included in an image represented by the frame data from the frame data. However, the present invention is not limited thereto, and the neural network device 1630 may receive various types of input data according to the type or function of the electronic device on which the electronic system 1600 is mounted, and generates a recognition signal according to the input data. can do.

The memory 1640 is a storage location for storing data, and may store an operating system (OS), various programs, and various data. In an embodiment, the memory 1640 may store intermediate results, such as an output feature map, generated in a process of performing an operation of the neural network device 1630 in the form of an output feature list or an output feature matrix. In an embodiment, the compressed output feature map may be stored in the memory 1640. In addition, the memory 1640 may store quantized neural network data used in the neural network apparatus 1630, for example, parameters, a weight map, or a weight list.

The memory 1640 may be DRAM, but is not limited thereto. The memory 1640 may include at least one of a volatile memory or a nonvolatile memory. Nonvolatile memory includes ROM, PROM, EPROM, EEPROM, flash memory, PRAM, MRAM, RRAM, FRAM, and the like. Volatile memory includes DRAM, SRAM, SDRAM, PRAM, MRAM, RRAM, FeRAM, and the like. In an embodiment, the memory 1640 may include at least one of HDD, SSD, CF, SD, Micro-SD, Mini-SD, xD, or Memory Stick.

The sensor module 1650 may collect information around an electronic device in which the electronic system 1600 is mounted. The sensor module 1650 may sense or receive a signal (eg, an image signal, an audio signal, a magnetic signal, a biometric signal, a touch signal, etc.) from the outside of the electronic device and convert the sensed or received signal into data. To this end, the sensor module 1650 is a sensing device such as a microphone, an imaging device, an image sensor, a light detection and ranging (LIDAR) sensor, an ultrasonic sensor, an infrared sensor, a bio sensor, and a touch sensor. It may include at least one of.

The sensor module 1650 may provide the converted data to the neural network device 1630 as input data. For example, the sensor module 1650 may include an image sensor, and generates a video stream by photographing an external environment of the electronic device, and sends a continuous data frame of the video stream to the neural network device 1630 as input data. Can be provided in order. However, the present invention is not limited thereto, and the sensor module 1650 may provide various types of data to the neural network device 1630.

The communication module 1660 may include various wired or wireless interfaces capable of communicating with external devices. For example, the communication module 1660 includes a wireless local area network (WLAN) such as a wired local area network (LAN), a wireless fidelity (Wi-fi), and a wireless personal communication network such as Bluetooth. Personal Area Network; WPAN), wireless USB (Wireless Universal Serial Bus), Zigbee, NFC (Near Field Communication), RFID (Radio-frequency identification), PLC (Power Line communication), or 3G (3rd Generation), 4G (4th) Generation), LTE (Long Term Evolution), and the like, a communication interface accessible to a mobile cellular network.

Hereinafter, the neural network device 1630 will be described in more detail with reference to FIG. 17.

17 is a block diagram illustrating an example of a configuration of a neural network device.

The neural network apparatus 1700 may be implemented as various types of devices such as a personal computer (PC), a server device, a mobile device, and an embedded device. For example, the neural network device 1700 is a smartphone, tablet device, Augmented Reality (AR) device, Internet of Things (IoT) device, and autonomous driving that perform voice recognition, image recognition, and image classification using a neural network. It may be implemented in automobiles, robotics, medical devices, etc., but is not limited thereto. Furthermore, the neural network apparatus 1700 may correspond to a dedicated hardware accelerator (HW accelerator) mounted in the above device, and the neural network apparatus 1700 is a neural processing unit (NPU), which is a dedicated module for driving a neural network, It may be a hardware accelerator such as a Tensor Processing Unit (TPU) or a Neural Engine, but is not limited thereto.

Referring to FIG. 17, the neural network device 1700 includes a processor 1710 and a memory 1730. In the neural network device 1700 illustrated in FIG. 17, only components related to the present embodiments are shown. Therefore, it is apparent to those skilled in the art that the neural network apparatus 1700 may further include other general-purpose components in addition to the components shown in FIG. 17.

The processor 1710 may quantize a parameter including at least one of input activation values and weight values processed in a layer of a neural network.

First, the processor 1710 may determine a first quantization value by performing log quantization on a parameter.

Also, the processor 1710 may compare an error and a threshold value between a first inverse quantization value and a parameter obtained by dequantization of the first quantization value. In this case, the threshold may be determined based on a trade-off relationship between a recognition rate of a neural network and a size of data according to quantization of a parameter.

When the error as a result of the comparison is greater than the threshold value, the processor 1710 may determine the second quantization value by performing log quantization on the error. Meanwhile, when the error is less than the threshold value, the parameter may be quantized to the first quantization value.

Finally, the processor 1710 may quantize the parameter into a value obtained by grouping the first quantization value and the second quantization value. In addition, the processor 1710 may add a tag bit to each of the first quantization value and the second quantization value to indicate that the first quantization value and the second quantization value are quantized values of one parameter. For example, the processor 1710 adds a first tag bit indicating that there is a second quantization value consecutive to the first quantization value, before the first bit or after the last bit among the bits indicating the first quantization value. , A second tag bit indicating that there is no consecutive quantization value in the second quantization value may be added before the first bit or after the last bit among bits indicating the second quantization value. Alternatively, the processor 1710 may include a first quantization value and a second quantization value in any one of the bits indicating the first quantization value before the first bit and after the last bit of the bits constituting the second quantization value You can also add a code value indicating that this is a continuous value.

Meanwhile, when the weight values processed in the layer of the neural network are quantized, the processor 1710 dequantizes the grouped value of the first quantization value and the second quantization value, and convolutions between the inverse quantized value and the input activation values. You can perform a convolution operation. For example, the processor calculates a first inverse quantization value that is a value obtained by inverse quantization of a first quantization value and a second inverse quantization value that is a value obtained by inverse quantization of the second quantization value, and calculates a first inverse quantization value and a second inverse quantization value. By adding the values, the grouped values may be inverse quantized.

The processor 1710 may be implemented with a central processing unit (CPU), a graphics processing unit (GPU), an application processor (AP), etc. included in the neural network device 1700, but is not limited thereto.

The memory 1730 may store data processed by the neural network device 1700 and data to be processed. For example, the memory 1730 may store a parameter including at least one of input activation values and weight values processed in a layer, a first quantization value, and a second quantization value. The memory 1730 includes a random access memory (RAM) such as dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and CD- ROM, Blu-ray or other optical disk storage, hard disk drive (HDD), solid state drive (SSD), or flash memory, but is not limited thereto.

Meanwhile, the above-described method can be written as a program that can be executed on a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium. In addition, the structure of the data used in the above-described method can be recorded on a computer-readable recording medium through various means. The computer-readable recording medium includes a storage medium such as a magnetic storage medium (e.g., ROM, RAM, USB, floppy disk, hard disk, etc.), and an optical reading medium (e.g., CD-ROM, DVD, etc.). do.

We have looked at the center of the preferred embodiments so far. Those of ordinary skill in the art to which the present invention pertains will appreciate that it may be implemented in a modified form without departing from the essential characteristics of the above-described substrate. Therefore, the disclosed embodiments should be considered from a descriptive point of view rather than a limiting point of view, and the scope of the rights is indicated in the claims rather than the above description, and should be interpreted as including all differences within the scope equivalent thereto.

Claims (19)

In a method of quantizing parameters in a neural network,
Determining a first quantization value by performing log quantization on a parameter including at least one of input activation values and weight values processed in a layer of the neural network;
Comparing an error and a threshold value between a first inverse quantization value obtained by dequantization of the first quantization value and the parameter;
Determining a second quantization value by performing log quantization on the error when the error is greater than the threshold value as a result of the comparison; And
And quantizing the parameter into a grouped value of the first quantization value and the second quantization value. The method of claim 1,
The step of determining the first quantization value,
The first quantization value is determined by performing log quantization on a value corresponding to a quantization level closest to the parameter among the plurality of quantization levels,
The step of determining the second quantization value,
A method of determining the second quantization value by performing log quantization on a value corresponding to a quantization level closest to the error among the plurality of quantization levels. The method of claim 1,
The second quantization value is,
The method expressed by the number of bits equal to the number of bits representing the first quantization value. The method of claim 1,
The threshold is,
The method is determined based on a trade off relationship between the recognition rate of the neural network and the size of data according to quantization of the parameter. The method of claim 1,
The quantizing step,
The method further comprising adding a tag bit to each of the first quantization value and the second quantization value. The method of claim 5,
The step of adding,
A first tag bit indicating that the second quantization value is present in the first quantization value is added before or after the first bit of the bits indicating the first quantization value,
A method of adding a second tag bit indicating that there is no consecutive quantization value to the second quantization value before or after the first bit of the bits indicating the second quantization value. The method of claim 1,
The quantizing step,
The first quantization value and the second quantization value are consecutive in any one of a first bit of bits representing the first quantization value and a last bit of the bits constituting the second quantization value. The method further comprising adding a code value indicating that it is a value. The method of claim 1,
Inverse quantizing the grouped values; And
And performing a convolution operation between a value obtained by inverse quantization of the grouped value and the input activation values. The method of claim 8,
Inverse quantizing the grouped values,
Calculating each of a first inverse quantization value that is a value obtained by inverse quantization of the first quantization value and a second inverse quantization value that is a value obtained by inverse quantization of the second quantization value; And
And obtaining an inverse quantization value from the grouped value by adding the first inverse quantization value and the second inverse quantization value. A computer-readable recording medium on which one or more programs including instructions for executing the method of any one of claims 1 to 9 are recorded. In a device implementing a neural network,
Memory; And
A processor that quantizes a parameter including at least one of input activation values and weight values processed in a layer of a neural network,
The processor determines a first quantization value by performing log quantization on a parameter, and an error between the first inverse quantization value obtained by dequantization of the first quantization value and the parameter And a threshold value, and when the error is greater than the threshold value as a result of the comparison, a second quantization value is determined by performing log quantization on the error, and the parameter is the first quantization value and the second quantization value. A device that quantizes values into grouped values. The method of claim 11,
The processor,
The first quantization value is determined by performing log quantization on a value corresponding to a quantization level closest to the parameter among the plurality of quantization levels,
An apparatus for determining the second quantization value by performing log quantization on a value corresponding to a quantization level closest to the error among the plurality of quantization levels. The method of claim 11,
The second quantization value is,
The device expressed by the number of bits equal to the number of bits representing the first quantization value. The method of claim 11,
The threshold is,
The apparatus is determined based on a trade-off relationship between the recognition rate of the neural network and the size of data according to quantization of the parameter. The method of claim 11,
The processor,
An apparatus for adding a tag bit to each of the first quantization value and the second quantization value. The method of claim 15,
The processor,
A first tag bit indicating that the second quantization value is present in the first quantization value is added before or after the first bit of the bits indicating the first quantization value,
Apparatus for adding a second tag bit indicating that there is no consecutive quantization value to the second quantization value before or after the first bit of the bits indicating the second quantization value. The method of claim 11,
The processor,
The first quantization value and the second quantization value are consecutive in any one of a first bit of bits representing the first quantization value and a last bit of the bits constituting the second quantization value. A device that adds a code value indicating that it is a value. The method of claim 11,
The processor,
Inverse quantization of the grouped values,
An apparatus for performing a convolution operation between a value obtained by inverse quantization of the grouped value and the input activation values. The method of claim 18,
The processor,
Each of a first inverse quantization value that is a value obtained by inverse quantization of the first quantization value and a second inverse quantization value that is a value obtained by inverse quantization of the second quantization value is calculated,
An apparatus for obtaining an inverse quantization value from the grouped value by adding the first inverse quantization value and the second inverse quantization value.

Images