KR20210035702A - Method of artificial neural network quantization and method of computation using artificial neural network - Google Patents

Abstract

According to a technical idea of the present disclosure, a computing system comprises a neural network system for driving an artificial neural network and a quantization system for quantizing the artificial neural network. The quantization system can quantize parameters of an artificial neural network to generate quantized parameters of the artificial neural network, generate a quantization error of parameters of the artificial neural network based on the parameters of the artificial neural network and the quantized parameters, generate a correction bias based on the quantized parameters and the quantization error of the parameters of the artificial neural network, and transmit the generated quantized parameters and the correction bias to the neural network system.

Description

Quantization method of artificial neural network and computation method using artificial neural network {METHOD OF ARTIFICIAL NEURAL NETWORK QUANTIZATION AND METHOD OF COMPUTATION USING ARTIFICIAL NEURAL NETWORK}

The technical idea of the present disclosure relates to a quantization method of an artificial neural network and an operation method using an artificial neural network. Specifically, an expected value of an error occurring in the quantization process is generated as a correction bias, and the generated correction bias is a quantized artificial neural network. The present invention relates to a quantization method of an artificial neural network that is reflected in the result of an operation through the method and an operation method using an artificial neural network.

An artificial neural network may refer to a computing device or a method performed by a computing device to implement interconnected sets of artificial neurons (or neuronal models). Artificial neurons can generate output data by performing simple operations on the input data, and the output data can be passed on to other artificial neurons. As an example of an artificial neural network, a deep neural network or deep learning may have a multi-layer structure.

The technical idea of the present disclosure is a quantization method of an artificial neural network and a computation method using an artificial neural network in which an expected value of an error occurring in the quantization process is generated as a correction bias and the generated correction bias is reflected in the calculation result through a quantized artificial neural network Provides.

In order to achieve the above object, a computing system according to an aspect of the technical idea of the present disclosure includes a neural network system for driving an artificial neural network and a quantization system for quantizing the artificial neural network, and the quantization system includes parameters of the artificial neural network. By quantizing them, quantized parameters of the artificial neural network are generated, quantization errors of the parameters of the artificial neural network are generated based on the parameters of the artificial neural network and the quantized parameters, and quantization errors of the quantized parameters and the parameters of the artificial neural network are calculated. A correction bias may be generated based on the generated quantized parameters and a correction bias may be transmitted to the neural network system.

A computation method using an artificial neural network according to an aspect of the technical idea of the present disclosure includes quantizing weights and biases of the artificial neural network, correcting the quantized bias to include an error due to quantization, and quantizing an input sample. , Performing a first multiply-accumulate (MAC) operation based on the quantized weight of the artificial neural network and the quantized input sample, and reflecting the corrected quantized bias to the result of the first MAC operation. .

A quantization method of an artificial neural network according to an aspect of the technical idea of the present disclosure includes quantizing parameters of the artificial neural network, calculating a quantization error of parameters based on parameters of the artificial neural network and quantized parameters, and quantized parameters. And generating a correction bias based on the quantization error of the parameters and parameters.

According to the quantization method of the artificial neural network and the computation method using the artificial neural network according to an embodiment of the present disclosure, an expected value of an error occurring in the quantization process is generated as a correction bias, and the generated correction bias is calculated through a quantized artificial neural network. By reflecting it in the result, it is possible to have a reduced complexity due to the use of a quantized artificial neural network and have good performance according to the reflection of a correction bias.

1 is a diagram illustrating an artificial neural network according to an embodiment of the present disclosure.
2 is a diagram illustrating a computing system according to an embodiment of the present disclosure.
3 is a flowchart illustrating operations of a neural network system, a parameter quantizer, a sample quantizer, and a bias corrector according to an embodiment of the present disclosure.
4 is a diagram for describing an architecture of a computational graph according to an embodiment of the present disclosure.
5 is a diagram for describing an operation of reflecting an expected value of a quantization error of a neural network system to a quantized operation according to an embodiment of the present disclosure.
6 is a diagram illustrating a method of generating a correction bias according to an embodiment of the present disclosure.
7 is a diagram illustrating a method of generating a correction bias according to an embodiment of the present disclosure.
8 is a diagram for describing an operation of reflecting an expected value of a quantization error of a neural network system to a quantized operation according to an embodiment of the present disclosure.
9 is a flowchart illustrating a method of determining a reference sample, a quantized reference sample, and a quantization error of a reference sample according to an embodiment of the present disclosure.
10 is a diagram illustrating a computing system according to an embodiment of the present disclosure.
11 is a diagram illustrating a method of predicting a next input sample according to an embodiment of the present disclosure.
12 is a diagram illustrating a method of predicting a next input sample according to an embodiment of the present disclosure.
13 is a flowchart illustrating a method of determining a reference sample, a quantized reference sample, and a quantization error of a reference sample according to an embodiment of the present disclosure.
14 is a diagram illustrating a method of performing an operation on an input sample of a next sequence through a quantized artificial neural network according to an embodiment of the present disclosure.
15 is a flowchart illustrating an operation method using an artificial neural network according to an embodiment of the present disclosure.
16 is a block diagram illustrating an electronic device according to an embodiment of the present disclosure.
17 is a block diagram illustrating an electronic device according to an embodiment of the present disclosure.

1 is a diagram illustrating an artificial neural network according to an embodiment of the present disclosure. Specifically, FIG. 1 is a diagram schematically illustrating a structure of a deep neural network 10 as an example of an artificial neural network according to an embodiment of the present disclosure.

An artificial neural network (ANN) may refer to a computing system focused on biological neural networks that make up an animal's brain. Unlike classical algorithms that perform tasks according to predefined conditions, such as rule-based programming, artificial neural networks (ANNs) perform tasks by considering multiple samples (or examples). You can learn to do it. An artificial neural network (ANN) may have a structure in which artificial neurons (or neurons) are connected, and a connection between neurons may be referred to as a synapse. Neurons can process the received signal and transmit the processed signal to other neurons through synapses. The output of a neuron may be referred to as “activation”. Neurons and/or synapses may have variable weights, and depending on the weights, the influence of signals processed by the neurons may increase or decrease. In particular, weights related to individual neurons may be referred to as bias.

A deep neural network (DNN) or a deep learning architecture may have a layer structure, and an output of a specific layer may be an input of a subsequent layer. In such a multi-layered structure, each of the layers may be trained according to a plurality of samples. Artificial neural networks such as deep neural networks can be implemented by a plurality of processing nodes each corresponding to artificial neurons, and high computational complexity may be required to obtain good results, such as high accuracy results. And, accordingly, many computing resources may be required.

Referring to FIG. 1, the deep neural network 10 may include a plurality of layers (L1, L2, L3, ..., LN), and the output of the layer may be input to a subsequent layer through at least one channel. have. For example, the first layer L1 may provide an output to the second layer L2 through a plurality of channels CH11...CH1x by processing the sample SAM, and the second layer L2 ) May also provide an output to the third layer L3 through a plurality of channels CH21...CH2y. Finally, the Nth layer LN may output the result RES, and the result RES may include at least one value related to the sample SAM. The number of channels through which the outputs of each of the plurality of layers L1, L2, L3, ..., LN are transmitted may be the same or different. For example, the number of channels CH21...CH2y of the second layer L2 and the number of channels CH31...CH3z of the third rater L3 may be the same or different. .

The sample SAM may be input data processed by the deep neural network 10. For example, the sample (SAM) may be an image including a character written by a human pen, and the deep neural network 10 may output a result (RES) including a value representing the character by recognizing the character from the image. have. The result RES may include a plurality of probabilities corresponding to different letters, and the most powerful letter among different letters may correspond to the highest probability. Each of the plurality of layers (L1, L2, L3, ..., LN) of the deep neural network 10 is a sample (SAM) based on values generated by learning a plurality of images including letters, such as weights, biases, etc. And it can generate its own outputs by processing the output of the previous layer.

Meanwhile, the deep neural network 10 may include a large number of layers or channels according to an exemplary embodiment, and accordingly, the computational complexity of the deep neural network 10 may increase. The deep neural network 10 with high computational complexity may require a lot of resources. Therefore, in order to reduce the computational complexity of the deep neural network 10, the deep neural network 10 may be quantized. Quantization of the deep neural network 10 refers to a process of mapping input values to values smaller than the number of input values, such as mapping a real number to an integer through rounding. can do. Meanwhile, the quantized deep neural network 10 may have low computational complexity, but may have reduced accuracy due to an error occurring in a quantization process.

As will be described later with reference to the following drawings, the quantized deep neural network 10 according to an embodiment of the present disclosure may reflect an expected value of an error occurring in the quantization process in the calculation result, thereby improving performance. While having, it can have a reduced complexity.

2 is a diagram illustrating a computing system according to an embodiment of the present disclosure.

Referring to FIG. 2, the computing system 1000 may include a quantization system 100 and a neural network system 200. The neural network system 200 may provide an artificial neural network, and the quantization system 100 may quantize an artificial neural network provided from the neural network system 200, and provide an at least partially quantized artificial neural network to the neural network system 200 can do. In FIG. 1, the neural network system 200 and the quantization system 100 are illustrated as being separated, but according to an embodiment, the neural network system 200 and the quantization system 100 may be implemented as one system.

The neural network system 200 may be any system that provides (or drives) an artificial neural network, and may be referred to as a neural network device. For example, the neural network system 200 may be a computing system including at least one processor and a memory. As a non-limiting example, the neural network system 200 may be a stationary computing system such as a desktop computer and a server, as well as a mobile computing system such as a laptop computer and a smart phone.

In an embodiment, the neural network system 200 may drive an artificial neural network and may provide information about the artificial neural network to the quantization system 100. In an embodiment, the neural network system 200 may drive an artificial neural network according to information provided from the quantization system 100 and may provide information on the driven artificial neural network to the quantization system 100.

The quantization system 100 may be any system that performs quantization of an artificial neural network, and may be referred to as a quantization device. For example, the quantization system 100 may be a computing system including at least one processor and a memory. The quantization system 100 may be a fixed computing system or a mobile computing system. The quantization system 100 may quantize the artificial neural network based on information of the artificial neural network provided from the neural network system 200.

Referring to FIG. 2, the quantization system 100 may include a neural network interface 110, a parameter quantizer 120, a sample quantizer 130, and a bias corrector 140. In one embodiment, each of the neural network interface 110, the parameter quantizer 120, the sample quantizer 130, and the bias corrector 140 is a logic block implemented through logic synthesis, a software block performed by a processor, or It can be implemented in a combination of. In one embodiment, each of the neural network interface 110, the parameter quantizer 120, the sample quantizer 130, and the bias corrector 140 may be a procedure as a set of a plurality of instructions executed by a processor. Can be stored in memory accessible by

The neural network interface 110 may provide an interface to the neural network system 200 to the parameter quantizer 120 and the sample quantizer 130. For example, the neural network interface 110 may provide parameters of an artificial neural network received from the neural network system 200 to the parameter quantizer 120, and the quantized parameters received from the parameter quantizer 120 are used in the neural network system. It can be provided to (200). In addition, the neural network interface 110 may provide samples received from the neural network system 200 to the sample quantizer 130, and provide quantized samples received from the sample quantizer 130 to the neural network system 200 can do. In addition, the neural network interface 110 may provide the correction bias received from the bias corrector 140 to the neural network system 200.

The parameter quantizer 120 may generate quantized parameters from parameters received from the neural network system 200 through the neural network interface 110. For example, the parameter quantizer 120 may receive weights and biases as parameters of an artificial neural network, and generate quantized weights and quantized biases. The parameter quantizer 120 may provide the quantized weight to the neural network system 200 through the neural network interface 110.

In addition, the parameter quantizer 120 may generate a quantization error of the weight and a quantization error of the bias using the quantized weight and the quantized bias. The quantization error of the weight may include an error generated in the process of quantizing the weight of the artificial neural network. The parameter quantizer 120 may generate a quantization error of the weight from the received weight and the quantized weight by using a characteristic that the unquantized weight is equal to the sum of the quantized weight and the quantization error of the weight. In addition, the bias quantization error may include an error generated in the process of quantizing the bias of the artificial neural network. Using a characteristic that the non-quantized bias is equal to the sum of the quantized bias and the quantization error of the bias, the parameter quantizer 120 may generate a quantization error of the bias from the received bias and the quantized bias.

In addition, the parameter quantizer 120 may provide information used to generate a correction bias for correcting an error due to quantization to the bias corrector 140. For example, the parameter quantizer 120 may provide a quantized weight, a quantized bias, a quantization error of the weight, and a quantization error of the bias to the bias corrector 140.

The sample quantizer 130 may generate quantized samples from samples received from the neural network system 200 through the neural network interface 110. For example, the sample quantizer 130 may receive a plurality of images, voice data, and the like, and generate quantized images, quantized voice data, and the like. The sample quantizer 130 may provide quantized samples to the neural network system 200 through the neural network interface 110. Meanwhile, since parameters and samples have different characteristics in an artificial neural network, quantization of parameters and quantization of samples can be separated.

The bias corrector 140 may generate a correction bias for correcting an error due to quantization using information received from the parameter quantizer 120. In an embodiment, the bias corrector 140 may generate a correction bias by correcting a quantized bias to include an error due to quantization. Examples of the operation of the bias corrector 140 for generating a correction bias will be described later with reference to FIGS. 6 and 7. The bias corrector 140 may provide the generated correction bias to the neural network system 200 through the neural network interface 110.

The neural network system 200 may perform a multiply-accumulate (MAC) operation based on the quantized weight received from the parameter quantizer 120 and the quantized sample received from the sample quantizer 130. In addition, the neural network system 200 may generate a final operation result by reflecting the correction bias received from the bias corrector 140 on the MAC operation result.

3 is a flowchart illustrating operations of a neural network system, a parameter quantizer, a sample quantizer, and a bias corrector according to an embodiment of the present disclosure. Specifically, FIG. 3 illustrates a quantization operation of the artificial neural network of the neural network system 200, the parameter quantizer 120, the sample quantizer 130, and the bias corrector 140 of FIG. 2 and an operation operation using the quantized artificial neural network. It is a flow chart showing.

2 and 3, the neural network system 200 may include an artificial neural network (S100). In addition, the neural network system 200 may provide a weight W and a bias as parameters of the artificial neural network to the parameter quantizer 120 together with the quantization request (S105). In addition, the parameter quantizer 120 may quantize the received weight W and bias (S110). In addition, the parameter quantizer 120 may provide the quantized weight q_W to the neural network system 200 (S110). The neural network system 200 may store the received quantized weight (q_W). In addition, the parameter quantizer 120 may provide a quantized weight (q_W), a quantized bias (q_bias), a quantization error of the weight (e_W), and a quantization error of the bias (e_bias) to the bias corrector 140 (S120). ). Meanwhile, the order of operations of steps S110 and S120 may be changed.

Further, the bias corrector 140 may generate a correction bias for correcting an error due to quantization based on the received information (S125). Specifically, the bias corrector 140 may generate a correction bias based on the received quantized weight (q_W), a quantized bias (q_bias), a quantization error of the weight (e_W), and a quantization error of the bias (e_bias). . Examples of the operation of the bias corrector 140 for generating a correction bias will be described later with reference to FIGS. 6 and 7. In addition, the bias corrector 140 may provide the generated correction bias q_bias1 to the neural network system 200 (S130). The neural network system 200 may store the received correction bias q_bias1.

The neural network system 200 may receive a new sample X (S135). In addition, the neural network system 200 may provide the sample X to the sample quantizer 130 together with the quantization request (S140). In addition, the sample quantizer 130 may quantize the received sample X (S145). In addition, the sample quantizer 130 may provide the quantized sample q_X to the neural network system 200 (S150). The neural network system 200 may perform a MAC operation based on the quantized weight (q_W) and the quantized sample (q_X) (S155). In addition, the neural network system 200 may generate a final calculation result by reflecting the correction bias q_bias1 in the MAC calculation result (S160).

4 is a diagram for describing an architecture of a computational graph according to an embodiment of the present disclosure.

Referring to FIG. 4, the calculation graph 20 is a graph representing a mathematical model expressed using nodes and edges. The architecture of the calculation graph 20 may correspond to the architecture of an artificial neural network or a quantized artificial neural network. For example, an artificial neural network or a quantized artificial neural network may be implemented as a convolutional neural network (CNN), but the present disclosure is not limited thereto. When the quantized artificial neural network of FIG. 4 represents a convolutional neural network, the calculation graph 20 may correspond to some layers of the convolutional neural network. For example, the calculation graph 20 may correspond to one layer that performs MAC operations, such as a convolutional neural network, a convolutional layer, and a fully connected layer. Hereinafter, for convenience of explanation, on the assumption that the calculation graph 20 is a convolution layer, a method in which the neural network system 200 of FIG. 2 performs a MAC operation using quantized weights and quantized samples will be described. .

2 and 4, the neural network system 200 provides the weight (W) of the artificial neural network together with the quantization request to the parameter quantizer 120, and the quantized weight (q_W) from the parameter quantizer 120 Can be received. In addition, the neural network system 200 may provide the received input sample X together with a quantization request to the parameter quantizer 120 and receive the quantized input sample q_X from the parameter quantizer 120. In addition, the neural network system 200 performs a MAC operation based on the quantized weight (q_W) and the quantized input sample (q_X), and reflects a bias (not shown) in the MAC operation result to obtain a quantized output sample (q_Y). Can be generated.

The quantized input sample (q_X) and the quantized output sample (q_Y) may be a two-dimensional or higher-dimensional matrix, and may have respective activation parameters. If the quantized input sample (q_X) and the quantized output sample (q_Y) correspond to, for example, a three-dimensional matrix, the quantized input sample (q_X) and the quantized output sample (q_Y) are the width (W) (or column ), height (H) (or low), and depth (D). In this case, the depth D may be referred to as the number of channels.

In the convolutional layer, a convolution operation may be performed on a quantized input sample (q_X) and a quantized weight (q_W), and as a result, a quantized output sample (q_Y) may be generated. The quantized weight q_W may filter the quantized input sample q_X, and may be referred to as a filter or a kernel. The quantized weight (q_W) may have a kernel size (K) (i.e., the size of the weight), and the depth of the quantized weight (q_W), that is, the number of channels of the quantized weight (q_W), is the quantized input sample (q_X Can be equal to the depth of ). The quantized weight q_W may be shifted by traversing the quantized input sample q_X as a sliding window. During each shift, each of the weights included in the quantized weight q_W may be multiplied and added to all values in the overlapped region with the quantized input sample q_X. As the quantized input sample q_X and the quantized weight q_W are convolved, one channel of the quantized output sample q_Y may be generated. Although one quantized weight (q_W) is shown in FIG. 1, in fact, a plurality of quantized weights (q_W) are convolved with the quantized input sample (q_X), and a plurality of channels of the quantized output sample (q_Y) Can be created.

Meanwhile, performing a MAC operation using a quantized artificial neural network and a quantized input may cause an error compared to a case of performing a MAC operation using an unquantized artificial neural network and a non-quantized input.

Specifically, the computation of the artificial neural network can be expressed as the following equation.

는 양자화된 가중치(w),

는 가중치(W)의 양자화 오차,

는 양자화된 바이어스(bias),

는 바이어스(bias)의 양자화 오차,

는 양자화된 입력 샘플(X),

는 입력 샘플(X)의 양자화 오차)(C is the number of input channels, K is the kernel size,

Is the quantized weight (w),

Is the quantization error of the weight (W),

Is the quantized bias,

Is the quantization error of the bias,

Is the quantized input sample (X),

Is the quantization error of the input sample (X))

In addition, the computation of the artificial neural network can be divided into a quantized operation (①) and a quantization error (②). The conventional neural network system 200 quantizes the weight (W) and the input sample (X) without direct consideration of the quantization error (②) (that is, without reflection of the quantization error (②) in the computation of the artificial neural network). We focused on implementing the optimized quantized operation (①) to reduce the error. On the other hand, the neural network system 200 according to an embodiment of the present disclosure calculates the expected value of the quantization error (②) and reflects the calculated expected value to the quantized operation (①), thereby increasing the accuracy of the quantized artificial neural network. You can increase it.

Meanwhile, in FIG. 4, for convenience of explanation, operation in one layer of the artificial neural network is illustrated and described, but the operation of the artificial neural network according to the technical idea of the present disclosure is substantially performed for each of a plurality of layers constituting the artificial neural network. It can be applied in the same way.

5 is a diagram for describing an operation of reflecting an expected value of a quantization error of a neural network system to a quantized operation according to an embodiment of the present disclosure.

2, 4, and 5, the neural network system 200 according to an embodiment of the present disclosure is based on a quantized input sample (q_X) and a quantized weight (q_W) received from the quantization system 100. MAC operation can be performed. In addition, the neural network system 200 reflects a correction bias (q_bias1) generated by the bias corrector 140 of the quantization system 100 to the MAC operation result, instead of a bias (q_bias) simply quantized the bias of the artificial neural network, A quantized output sample (q_Y) can be generated. The correction bias q_bias may have the same size as the size of the MAC operation result, as described later in FIGS. 6 and 7. Therefore, the correction bias (q_bias) can be added to all values in the region overlapping with the result of the MAC operation. Meanwhile, in FIG. 5, quantized output samples q_Y for an arbitrary channel (channel k) are displayed, but in practice, quantized output samples q_Y for each channel may be generated.

The bias corrector 140 may generate a correction bias q_bias1 including a quantization error as well as a quantized bias q_bias of the artificial neural network. Specifically, the bias corrector 140 may generate a correction bias q_bias1 to include the equation ③ in the quantization equation of the artificial neural network below.

,

,

, 및

는 샘플 양자화기(130)의 인공 신경망의 양자화를 수행함으로써 미리 알고 있는 값이다. 그러나

및

의 경우, 입력 샘플(X)에 대한 연산 처리를 수행하는 시점에 미리 알 수 없는 값이다. 따라서, 바이어스 보정기(140)는 실제 입력 샘플(X)에 대한

및

이 아닌, 실제 입력 샘플(X)을 대신하는 참조 샘플(X’)의

및

를 이용하여 보정 바이어스(q_bias1)를 생성할 수 있다.In the above artificial neural network's quantization equation,

,

,

, And

Is a value known in advance by performing quantization of the artificial neural network of the sample quantizer 130. But

And

In the case of, it is a value that is not known in advance at the time when the operation processing on the input sample (X) is performed. Therefore, the bias corrector 140 is used for the actual input sample (X).

And

Of the reference sample (X') instead of the actual input sample (X).

And

A correction bias (q_bias1) can be generated by using.

를 생성하고, 복수의 샘플들의 양자화 오차들을 이용하여

를 생성할 수 있다. 그리고 바이어스 보정기(140)는 생성한

및

를 이용하여 보정 바이어스(q_bias1)를 생성할 수 있다. 복수의 샘플들을 이용하여 보정 바이어스(q_bias1)를 생성하는 구체적인 동작은 도 9와 관련하여 후술한다.For example, the sample quantizer 130 may generate a plurality of quantized samples and quantization errors of a plurality of samples by quantizing a plurality of samples selected from a sample pool. The bias corrector 140 uses a plurality of quantized samples

And using the quantization errors of a plurality of samples

Can be created. And the bias corrector 140 generated

And

A correction bias (q_bias1) can be generated by using. A specific operation of generating the correction bias q_bias1 using a plurality of samples will be described later with reference to FIG. 9.

를 생성하고, 이미 처리한 입력 샘플들의 양자화 오차들을 이용하여 다음 순서의 입력 샘플의

를 생성할 수 있다. 그리고 바이어스 보정기(140)는 생성한

및

를 이용하여 보정 바이어스(q_bias1)를 생성할 수 있다. 이미 처리한 입력 샘플들을 이용하여 보정 바이어스(q_bias1)를 생성하는 구체적인 동작은 도 10 내지 도 14와 관련하여 후술한다.As another example, the bias corrector 140 uses input samples that have already been processed through an artificial neural network.

And using the quantization errors of the input samples that have already been processed,

Can be created. And the bias corrector 140 generated

And

A correction bias (q_bias1) can be generated by using. A specific operation of generating the correction bias q_bias1 using input samples that have already been processed will be described later with reference to FIGS. 10 to 14.

6 is a diagram illustrating a method of generating a correction bias according to an embodiment of the present disclosure. Specifically, FIG. 6 is a diagram illustrating a method of generating a correction bias including a quantized bias and a quantization error of an artificial neural network.

The calculation of the correction bias can be expressed as the following equation.

(E is the expected value)

는 실제 입력 샘플(X)이 아닌 참조 샘플(X’)을 의미할 수 있고,

는 참조 샘플(X’)의 이용하여 생성할 수 있다. 참조 샘플(X’)을 결정하는 방법, 참조 샘플의 양자화 오차(

)를 산출하는 방법은 도 9 및 도 13에서 후술한다.In the last expression above,

May mean a reference sample (X'), not an actual input sample (X),

Can be generated using the reference sample (X'). How to determine the reference sample (X'), the quantization error of the reference sample (

A method of calculating) will be described later in FIGS. 9 and 13.

Referring to FIG. 6, the last equation above may be implemented by the operation of the bias corrector 140. The bias corrector 140 may perform a first MAC operation based on a quantization error (e_X') of a reference sample and a quantized weight (q_W). In addition, the bias corrector 140 may perform a second MAC operation based on the reference sample X'and the quantization error e_W of the weight. In addition, the bias corrector 140 adds the first MAC operation result, the second MAC operation result, and the bias of the artificial neural network (which is equal to the sum of the bias, quantized bias (q_bias), and quantization error (e_bias) of the bias) to correct a correction bias ( q_bias1) can be created.

7 is a diagram illustrating a method of generating a correction bias according to an embodiment of the present disclosure. Specifically, FIG. 7 is a diagram illustrating a deformable embodiment of FIG. 6.

The calculation of the correction bias of FIG. 6 may be expressed by the following equation.

는 인공 신경망의 양자화되지 않은 가중치(W)를 의미할 수 있다.

및

는 실제 입력 샘플(X)이 아닌 참조 샘플(X’)의 이용하여 생성할 수 있다. 참조 샘플(X’)을 결정하는 방법, 양자화된 참조 샘플(

) 및 참조 샘플의 양자화 오차(

)를 산출하는 방법은 도 9 및 도 13에서 후술한다.In the last expression above,

May mean an unquantized weight (W) of the artificial neural network.

And

Can be generated by using the reference sample (X'), not the actual input sample (X). A method of determining a reference sample (X'), a quantized reference sample (

) And the quantization error of the reference sample (

A method of calculating) will be described later in FIGS. 9 and 13.

Referring to FIG. 7, the last equation above may be implemented by the operation of the bias corrector 140. The bias corrector 140 may perform a third MAC operation based on a quantization error (e_X') of a reference sample and an unquantized weight (W). In addition, the bias corrector 140 may perform a fourth MAC operation based on the quantized reference sample q_X' and the quantization error e_W of the weight. In addition, the bias corrector 140 may generate a correction bias q_bias1 by adding the third MAC operation result, the fourth MAC operation result, and a bias of the artificial neural network.

The correction biases q_bias1 generated in FIGS. 6 and 7 may have different generation methods, but may have the same value. The quantized input sample (q_X) of FIG. 5, the reference sample (X′) of FIGS. 6 and 7, the quantization error (e_X′) of the quantized reference sample (q_X′), and the reference sample are all the same width (W) and They have the same height H, and the quantized weight q_W of FIG. 5, the quantized weight q_W of FIGS. 6 and 7 and the quantization error e_W of the weight may have the same kernel size K. Accordingly, the size of the correction bias q_bias1 generated in FIGS. 6 and 7 may be the same as the size of the MAC operation result of the quantized weight q_W and the quantized input sample q_X in one channel of FIG. 5. . Accordingly, the generated correction bias q_bias1 may be added to all values in a region overlapping with the result of the MAC operation of the quantized weight q_W and the quantized input sample q_X.

8 is a diagram for describing an operation of reflecting an expected value of a quantization error of a neural network system to a quantized operation according to an embodiment of the present disclosure. Specifically, FIG. 8 is a diagram illustrating a deformable embodiment of FIG. 5.

As described above with reference to FIG. 7, the correction bias q_bias1 may have the same size of the MAC operation result of the quantized weight q_W and the quantized input sample q_X in one channel. Meanwhile, according to a deformable embodiment, the correction bias q_bias1 may have a scalar value instead of a 2D matrix structure. In an embodiment, the bias corrector 140 may generate a second correction bias q_bias2 having a scalar value by taking a mean of values constituting the correction bias q_bias1. Meanwhile, a method of generating the second correction bias q_bias2 having a scalar value by the bias corrector 140 is not limited to the above-described example, and various methods may be applied.

The bias corrector 140 may provide the generated second correction bias q_bias2 to the neural network system 200. Meanwhile, according to an embodiment, the bias corrector 140 provides an existing correction bias (q_bias1) to the neural network system 200, and has a scalar value using the correction bias (q_bias1) received by the neural network system 200. It goes without saying that the second correction bias (q_bias2) can be generated.

Referring to FIG. 8, the neural network system 200 may perform a MAC operation based on a quantized input sample (q_X) and a quantized weight (q_W). In addition, the neural network system 200 may generate a quantized output sample q_Y by reflecting the second correction bias q_bias2 to the MAC operation result instead of the bias q_bias obtained by simply quantizing the bias of the artificial neural network. In this way, when the second correction bias (q_bias2) is used instead of the conventional correction bias (q_bias1), the efficiency of the storage space of the memory can be increased, and the calculation speed increases as only one scalar value is reflected in the MAC operation result. can do.

9 is a flowchart illustrating a method of determining a reference sample, a quantized reference sample, and a quantization error of a reference sample according to an embodiment of the present disclosure. Specifically, FIG. 9 is a method of determining a reference sample, a quantized reference sample, and a quantization error of the reference sample in order to generate a correction bias that can be fixedly used by the neural network system 200 and the quantization system 100 of FIG. 2 It is a flow chart showing.

2 and 9, the neural network system 200 may select a plurality of first samples from a sample pool (S210). Here, the plurality of first samples may be training samples used for training the artificial neural network. However, the present disclosure is not limited thereto, and the plurality of first samples may be samples not used for training of an artificial neural network, and may include both training samples used for training and samples not used for training.

In addition, the neural network system 200 may perform an operation on a plurality of first samples through an artificial neural network (S220). Specifically, the neural network system 200 may perform an operation on a plurality of first samples through a neural network that is not quantized.

In addition, the neural network system 200 may select at least one second sample from among the plurality of first samples based on statistical distributions of a plurality of output samples of each of the layers constituting the artificial neural network (S230). Specifically, the neural network system 200 can check the statistical distributions of the output samples of each of the layers constituting the unquantized artificial neural network, for example, the first layer (L1) to the n-th layer (Ln) of FIG. 1. have. Here, the statistical distributions may include at least one of average, variance, expected value, asymmetry, and kurtosis of the output samples, and are not limited to the above-described examples. In addition, the neural network system 200 may select at least one second sample from among the plurality of first samples based on statistical distributions. For example, the neural network system 200 may select at least one second sample corresponding to an output sample having a value close to the averages of each of the plurality of first samples. Meanwhile, the method of selecting at least one second sample based on statistical distributions is not limited to the above-described example.

In addition, the neural network system 200 may calculate a quantization error of the quantized second sample and the second sample using the selected second sample (S240). Specifically, the neural network system 200 may calculate the quantized second sample by performing an operation on the second sample through the quantized artificial neural network. In addition, by using the characteristics that the input sample (X) is the same as the quantized input sample (q_X) and the quantization error (e_X) of the input sample, the neural network system 200 performs a second sample from the second sample and the quantized second sample. The quantization error of can be calculated.

In addition, the quantization system 100 may determine a reference sample, a quantized reference sample, and/or a quantization error of the reference sample (S250). In one embodiment, the quantization system 100 calculates an average of each of the quantization errors of at least one second sample, the quantized second sample, and the second sample, the calculated average of the second sample, and the quantized second sample Each of the average and the average of the quantization error of the second sample may be determined as a reference sample, a quantized reference sample, and a quantization error of the reference sample. Meanwhile, the method for determining the quantization error of the reference sample, the quantized reference sample, and/or the reference sample by using the quantization error of the second sample from the second sample and the quantized second sample is described above. It is not limited to, and various methods can be applied.

In addition, the quantization system 100 may generate a correction bias based on the determined reference sample, the quantized reference sample, and/or a quantization error of the reference sample. Specifically, referring to FIG. 6, the bias corrector 140 of the quantization system 100 may perform a first MAC operation based on a quantization error (e_X') and a quantized weight (q_W) of a reference sample. In addition, the bias corrector 140 may perform a second MAC operation based on the reference sample X'and the quantization error e_W of the weight. In addition, the bias corrector 140 adds the first MAC operation result, the second MAC operation result, and the bias of the artificial neural network (which is equal to the sum of the bias, quantized bias (q_bias), and quantization error (e_bias) of the bias) to correct a correction bias ( q_bias1) can be created.

Alternatively, referring to FIG. 7, the bias corrector 140 of the quantization system 100 may perform a third MAC operation based on a quantization error (e_X') of a reference sample and an unquantized weight (W). In addition, the bias corrector 140 may perform a fourth MAC operation based on the quantized reference sample q_X' and the quantization error e_W of the weight. In addition, the bias corrector 140 may generate a correction bias q_bias1 by adding the third MAC operation result, the fourth MAC operation result, and a bias of the artificial neural network.

Meanwhile, the bias corrector 140 generates a second correction bias q_bias2 having a scalar value by taking an average of the values constituting the generated correction bias q_bias1 as described above with reference to FIG. 8. Of course you can.

In this way, the bias corrector 140 may determine a reference sample, a quantized reference sample, and/or a quantization error of the reference sample using a plurality of samples, and calculate the quantization error of the determined reference sample, the quantized reference sample, and the reference sample. A correction bias (q_bias1 or q_bias2) may be generated as a basis. The neural network system 200 may use the generated correction bias (q_bias1 or q_bias2) in the calculation of the input sample fixedly. Meanwhile, the present disclosure is not limited thereto, and according to an exemplary embodiment, the bias corrector 140 may periodically or aperiodically generate a new correction bias.

Meanwhile, the bias corrector 140 may determine a reference sample, a quantized reference sample, and a quantization error of the reference sample using input samples that have already been processed. A detailed description of this will be described later in FIGS. 10 to 14.

10 is a diagram illustrating a computing system according to an embodiment of the present disclosure. Specifically, FIG. 10 is a diagram illustrating a deformable embodiment of FIG. 2.

Referring to FIG. 10, the computing system 1000a may include a quantization system 100a and a neural network system 200. The quantization system 100a may generate a neural network interface 110, a parameter quantizer 120, a sample quantizer 130, a bias corrector 140a, and a sample generator 150a. The neural network system 200, the neural network interface 110, the parameter quantizer 120, and the sample quantizer 130 according to the embodiment of FIG. 10 are the neural network system 200, the neural network interface 110, and the parameter quantization of FIG. Since it may correspond to the group 120 and the sample quantizer 130, a redundant description will be omitted.

The sample generator 150a may be implemented as a logic block implemented through logic synthesis, a software block performed by a processor, or a combination thereof. On the other hand, in the illustration and description of FIG. 10, the sample generator 150a is illustrated and described as being included in the quantization system 100a, but according to the embodiment, the sample generator 150a is included in the neural network system 200 or , It may be implemented in a separate configuration from the neural network system 200 and the quantization system 100a.

The sample generator 150a may generate input samples in the next expected sequence using at least one input sample already processed by the neural network system 200. For example, the sample generator 150a may analyze at least one input sample that has already been processed, and predict the next input sample according to the analysis result. In addition, the sample generator 150a may provide the expected input sample to the bias corrector 140a. A detailed description of an operation of generating the next expected input sample using at least one input sample already processed by the sample generator 150a will be described later with reference to FIGS. 11 and 12.

The bias corrector 140a may generate a correction bias using the received expected input sample. A detailed method of generating a correction bias by the bias corrector 140a using an expected input sample will be described later in FIG. 11.

11 is a diagram illustrating a method of predicting a next input sample according to an embodiment of the present disclosure. Specifically, FIG. 11 is a diagram illustrating a method of predicting an input sample of a next sequence using at least one input sample already processed by the sample generator 150a of FIG. 10.

Input samples processed by the neural network system 200 may be continuously photographed images. Each of the consecutively photographed images has a small difference from the previous or subsequent images, and the difference between the images may have a directionality over time. Accordingly, the sample generator 150a may predict a next sequence of input samples by analyzing differences between input samples that have already been processed, for example, images that have been successively photographed. In addition, the sample generator 150a may provide an input sample of the next expected sequence to the bias corrector 140a. The sample generator 150a reads input samples that have already been processed from a memory (not shown) included in the quantization system 100a, or receives input samples that have already been processed from the neural network system 200 or any other distinct configuration. can do.

For example, referring to FIG. 11, the sample generator 150a includes a frame N-2 and a frame N-2, which are the most recent images among images already processed using a Kalman filter or the like. Motion vectors of an N-1 frame (Frame N-1) can be calculated. In addition, the sample generator 150a may identify a moving object (eg, a person) in the image using the calculated motion vectors. In addition, the sample generator 150a may generate an Expected Frame N by predicting the movement path of the identified object in the image.

Meanwhile, in the illustration and description of FIG. 11, the sample generator 150a is illustrated and described as predicting the next image using two recent images, but the present disclosure is not limited thereto, and two or more recent images are used. It goes without saying that the next image can be expected.

12 is a diagram illustrating a method of predicting a next input sample according to an embodiment of the present disclosure. Specifically, FIG. 12 is a diagram illustrating a method of predicting an input sample of a next sequence using at least one input sample already processed by the sample generator 150a of FIG. 10.

The sample generator 150a may include an artificial intelligence (AI) module 151a. The artificial intelligence module 151a may be an artificial intelligence module that has been trained to predict a next sample based on an input sample. The sample generator 150a may input at least one input sample that has already been processed to the artificial intelligence module 151a, and provide the output sample output from the artificial intelligence module 151 to the bias corrector 140a as the next input sample. have. The artificial intelligence module 151a may be implemented as a logic block implemented through logic synthesis, a software block executed by a processor, or a combination thereof.

On the other hand, with reference to FIGS. 11 and 12, the input samples processed by the neural network system 200 have been described as being continuously photographed images, but the present disclosure is not limited thereto, and various data such as voice data or depth data It can be a kind of data.

13 is a flowchart illustrating a method of determining a reference sample, a quantized reference sample, and a quantization error of a reference sample according to an embodiment of the present disclosure. Specifically, FIG. 13 shows a reference sample, a quantized reference sample, and a reference sample in order to generate a correction bias that the neural network system 200 and the quantization system 100a of FIG. 10 can use in the calculation of the next input sample. It is a flow chart showing a method of determining a quantization error.

10 and 13, the quantization system 100a may select at least one first sample from among input samples that have already been processed (S310 ). Specifically, the sample generator 150a of the quantization system 100a may select a preset number of first samples from among input samples that have already been processed. Here, the preset number may be set by a manufacturer or a user. For example, when the input samples are images continuously photographed, the sample generator 150a may select 100 recently processed images from among consecutive images that have already been processed as the first sample.

In addition, the quantization system 100a may predict a second sample in the next order based on the first sample (S320). Specifically, the sample generator 150a may analyze a preset number of first samples and predict a second sample in the next order according to the analysis result. A method of predicting the second sample in the next sequence by the sample generator 150a may be substantially the same as the method described above with reference to FIGS. 11 and 12. In addition, the sample generator 150a may provide the second sample to the bias corrector 140a.

In addition, the quantization system 100a may determine the expected second sample as a reference sample (S330). In addition, the quantization system 100a may determine a quantization error of the reference sample by using the quantization error of at least one first sample (S340). In an embodiment, the bias corrector 140a may determine the quantization error of the reference sample by taking an average of the quantization error of at least one first sample. Alternatively, the bias corrector 140a may check the quantization error of the most recently processed sample among the at least one first sample, and determine the determined quantization error as the quantization error of the reference sample. That is, the bias corrector 140a may determine the quantization error of the reference sample without using a characteristic that the input sample X is equal to the quantized input sample q_X and the quantization error e_X of the input sample.

In addition, the quantization system 100a may generate a correction bias based on the determined reference sample and a quantization error of the reference sample. Specifically, referring to FIG. 6, the bias corrector 140a of the quantization system 100a may perform a first MAC operation based on a quantization error (e_X') of a reference sample and a quantized weight (q_W). In addition, the bias corrector 140a may perform a second MAC operation based on the reference sample X'and the quantization error e_W of the weight. In addition, the bias corrector 140a adds the first MAC operation result, the second MAC operation result, and the bias of the artificial neural network (equal to the sum of the bias, quantized bias (q_bias), and quantization error (e_bias) of the bias) to compensate for the correction bias ( q_bias1) can be created.

Meanwhile, according to a deformable embodiment, the quantization system 100a may quantize the determined reference sample and generate a correction bias based on the quantized reference sample and a quantization error of the reference sample. Specifically, referring to FIG. 7, the bias corrector 140a of the quantization system 100a may perform a third MAC operation based on a quantization error (e_X′) of a reference sample and an unquantized weight (W). . In addition, the bias corrector 140a may perform a fourth MAC operation based on the quantized reference sample q_X' and the quantization error e_W of the weight. In addition, the bias corrector 140a may generate a correction bias q_bias1 by adding the third MAC operation result, the fourth MAC operation result, and a bias of the artificial neural network.

Meanwhile, the bias corrector 140a generates a second correction bias q_bias2 having a scalar value by taking an average of the values constituting the generated correction bias q_bias1 as described above with reference to FIG. 8. Of course you can.

In this way, the bias corrector 140a may determine the quantization error of the reference sample, the quantized reference sample, and/or the reference sample using input samples that have already been processed, and quantization of the determined reference sample, the quantized reference sample, and the reference sample. A correction bias (q_bias1 or q_bias2) may be generated based on the error. The neural network system 200 may use the generated correction bias (q_bias1 or q_bias2) in the calculation of input samples in the next sequence.

14 is a diagram illustrating a method of performing an operation on an input sample of a next sequence through a quantized artificial neural network according to an embodiment of the present disclosure. Specifically, FIG. 14 is a diagram illustrating a method for the quantization system 100a and the neural network system 200 of FIG. 10 to perform operations on input samples in the next order. Hereinafter, for convenience of description, it is assumed that input samples are images taken continuously.

Referring to FIGS. 10 and 14, consecutively photographed images (Frame 0 to Frame N), which are a plurality of input samples, may be sequentially processed. When processing for the N-1th frame is completed, the quantization system 100a may prepare to process the Nth frame.

Specifically, the quantization system 100a may select at least one image from among images that have already been processed (Frame 0 to Frame N-1). For example, referring to FIG. 14, the sample generator 150a of the quantization system 100a may select an N-1th frame (Frame N-1). In addition, the quantization system 100a may predict a next sequence of images based on the selected image. For example, referring to FIG. 14, the sample generator 150a predicts an Nth frame (Frame N) based on an N-1th frame (Frame N-1), and thus an expected Nth frame (Expected Frame N). Can be created.

In addition, the quantization system 100a may generate a correction bias using an image in the following order. For example, referring to FIG. 14, the bias corrector 140a receives an expected Nth frame N from the sample generator 150a, and refers to the received expected Nth frame N as a reference sample ( X'). In addition, the bias corrector 140a may determine the quantization error e_X' of the reference sample by using at least one quantization error among the already processed images Frame 0 to Frame N-1. In one embodiment, the bias corrector 140a determines the quantization error of the N-1th frame (Frame N-1) as the quantization error (e_X′) of the reference sample, or the most recently processed preset number of images. By taking the average of the quantization error, it is possible to determine the quantum and error (e_X') of the reference sample.

Quantization errors of the images (Frame 0 to Frame N-1) that have already been processed are calculated during the last operation on the images (Frame 0 to Frame N-1) that have already been processed, so that they can be stored in the memory 160. have. For example, through the characteristic that the input sample (X) is the same as the quantized input sample (q_X) and the quantization error (e_X) of the input sample, the quantization error of the already processed images (Frame 0 to Frame N-1) They may be calculated and stored in the memory 160. Therefore, the bias corrector 140a reads the quantization error of at least one of the images (Frame 0 to Frame N-1) already processed from the memory 160 and determines the quantization error of the reference sample using the read error. I can. Meanwhile, in FIG. 14, the memory 160 is illustrated and described as being included in the quantization system 100a, but the memory 160 is included in the neural network system 200, or the memory 160 is included in the neural network system 200 and the quantization system 100a. It can be implemented as a separate configuration.

In addition, the bias corrector 140a may generate a correction bias q_bias1 or q_bias2 based on the reference sample X'and the quantization error e_X' of the reference sample. In addition, the bias corrector 140a may provide the generated correction bias q_bias1 or q_bias2 to the neural network system 200 through the neural network interface 110.

In addition, the sample quantizer 130 quantizes the Nth frame (Frame N) to generate a quantized Nth frame (Quantized Frame N). In addition, the sample quantizer 130 may provide the quantized Nth frame N to the neural network system 200 through the neural network interface 110.

In addition, the neural network system 200 may perform an operation based on the received quantized frame N and a correction bias q_bias1 or q_bias2. Specifically, the neural network system 200 performs a MAC operation based on an Nth frame (Quantized Frame N) and a quantized weight, and reflects a correction bias (q_bias1 or q_bias2) in the MAC operation result to reflect the quantized output sample (q_Y). ) Can be created.

15 is a flowchart illustrating an operation method using an artificial neural network according to an embodiment of the present disclosure. Specifically, FIG. 15 is a flowchart illustrating an operation method using an artificial neural network of the computing systems 1000 and 1000a of FIG. 2 or 10.

2, 10, and 15, the computing systems 1000 and 1000a may quantize the parameters of the artificial neural network (S410). For example, the quantization systems 100 and 100a of the computing systems 1000 and 1000a may quantize parameters of an artificial neural network such as weights and biases. In addition, the computing systems 1000 and 1000a may generate a correction bias by correcting the quantized bias to include an error due to quantization (S420). Here, the correction bias may be generated using at least one of a quantized weight of an artificial neural network, a quantization error of a weight, a quantized bias, a quantization error of a bias, a reference sample, a quantized reference sample, and a quantization error of a reference sample. The reference sample may be determined through a plurality of samples different from the input sample currently being processed, or may be determined through at least one of input samples that have already been processed.

In addition, the computing systems 1000 and 1000a may quantize the input samples (S430). For example, the quantization systems 100 and 100a of the computing systems 1000 and 1000a may quantize the input samples. In addition, the quantization systems 100 and 100a may provide quantized input samples to the neural network system 200 of the computer systems 1000 and 1000a. Further, the computing systems 1000 and 1000a may perform a multiply-accumulate (MAC) operation based on the quantized weight and the quantized input sample (S440). For example, the neural network system 200 may receive a quantized input sample and perform a MAC operation based on the quantized weight and the received quantized input sample.

In addition, the computing systems 1000 and 1000a may reflect the correction bias for correcting the quantization error in the result of the MAC operation (S450). For example, the neural network system 200 of the computing systems 1000 and 1000a may generate a final operation result by reflecting the correction bias received from the quantization systems 100 and 100a on the result of the MAC operation. The computing system according to an embodiment of the present disclosure may generate an expected value of an error occurring in a quantization process as a correction bias, and reflect the generated correction bias to a result of a MAC operation through a quantized artificial neural network. Accordingly, a computation method using an artificial neural network according to an embodiment of the present disclosure may have a reduced complexity due to the use of a quantized artificial neural network and have good performance according to reflection of a correction bias.

16 is a block diagram illustrating an electronic device according to an embodiment of the present disclosure.

In an embodiment, the quantization systems 100 and 100a of FIG. 2 or 10 may be implemented with the electronic device 300 of FIG. 16. As shown in FIG. 16, the electronic device 300 may include a system memory 310, a processor 330, a storage 350, input/output devices 370, and communication connections 390. Components included in the electronic device 300 may be connected to each other to enable communication, for example, through a bus.

The system memory 310 may include a program 312. The program 312 may cause the processor 330 to perform quantization of an artificial neural network, quantization of an input sample, and generation of a correction bias according to embodiments of the present disclosure. For example, the program 312 may include a plurality of instructions executable by the processor 330. In addition, by executing a plurality of instructions included in the program 312 by the processor 330, the artificial neural network may be quantized, the input sample may be quantized, or a correction bias may be generated. As a non-limiting example, the system memory 310 may include a volatile memory such as static random access memory (SRAM) and dynamic random access memory (DRAM), and nonvolatile memory such as flash memory. It can also be included.

The processor 330 is capable of executing any instruction set (eg, IA-32 (Intel Architecture-32), 64-bit extended IA-32, x86-64, PowerPC, Sparc, MIPS, ARM, IA-64, etc.). It may include at least one core. The processor 330 may execute instructions stored in the system memory 310, and may perform quantization of an artificial neural network, quantization of input samples, or generation of a correction bias by executing the program 312.

The storage 350 may not lose stored data even when power supplied to the electronic device 300 is cut off. For example, the storage 350 includes non-volatile memory such as a Electrically Erasable Programmable Read-Only Memory (EEPROM), flash memory, Phase Change Random Access Memory (PRAM), and Resistance Random Access Memory (RRAM). , NFGM (Nano Floating Gate Memory), PoRAM (Polymer Random Access Memory), MRAM (Magnetic Random Access Memory), FRAM (Ferroelectric Random Access Memory). It may also include a storage medium such as a disk. In some embodiments, the storage 350 may be detachable from the electronic device 300.

In an embodiment, the storage 350 may store a program 312 for quantizing an artificial neural network, quantizing an input sample, and generating a correction bias according to an embodiment of the present disclosure. And before the program 312 is executed by the processor 330, the program 312 or at least a portion thereof from the storage 350 may be loaded into the system memory 310. In one embodiment, the storage 350 may store a file written in a program language, and a program 312 generated from a file by a compiler or the like, or at least a part thereof, may be loaded into the system memory 310.

In one embodiment, the storage 350 may store data to be processed by the processor 330 and/or data processed by the processor 330. For example, the storage 350 may store input samples, quantized input samples and quantization errors of the input samples, and may store a generated correction bias.

The input/output devices 370 may include an input device such as a keyboard and a pointing device, and may include an output device such as a display device and a printer. For example, the user may trigger the execution of the program 312 by the processor 330 through the input/output devices 370, input an input sample, and output an output sample and/or an error message. You can also check.

The communication connections 390 may provide access to a network outside the electronic device 300. For example, a network may include multiple computing systems and communication links, and the communication links may include wired links, optical links, wireless links, or any other type of links.

17 is a block diagram illustrating an electronic device according to an embodiment of the present disclosure. In an embodiment, the neural network system 200 of FIG. 2 or 10 may be implemented with the electronic device 400 of FIG. 17. The electronic device 400 is a non-limiting example, and may be any portable electronic device supplied with power through a battery or self-power, such as a mobile phone, a tablet PC, a wearable device, an IoT device, and the like.

17, the electronic device 400 may include a memory subsystem 410, input/output devices 430, a processing unit 450, and a network interface 470, and the memory subsystem 410 ), the input/output devices 430, the processing unit 450, and the network interface 470 may communicate with each other through the bus 490. In some embodiments, at least two or more of the memory subsystem 410, the input/output devices 430, the processing unit 450, and the network interface 470 are system-on-a-chip (System-on-a- Chip; SoC) can be included in one package.

Memory subsystem 410 may include RAM 412 and storage 414. The RAM 412 and/or the storage 414 may store instructions executed by the processing unit 450 and data to be processed. For example, the RAM 412 and/or the storage 414 may store parameters such as signals, weights, and biases of the artificial neural network. In some embodiments, storage 414 may include non-volatile memory.

The processing unit 450 may include a central processing unit (CPU) 452, a graphical processing unit (GPU) 454, a digital signal processor (DSP) 456, and a neural processing unit (NPU) 458. . Different from that shown in FIG. 17, in one embodiment, the processing unit 450 may include only at least some of the CPU 452, GPU 454, DSP 456 and NPU 458.

The CPU 452 directly performs a specific task in response to the overall operation of the electronic device 400, for example, an external input received through the input/output devices 430, or performs execution to other components of the processing unit 450. I can instruct. The GPU 454 may generate data for an image output through a display device included in the input/output devices 430 or may encode data received from a camera included in the input/output devices 430. DSP 456 may generate useful data by processing a digital signal, such as a digital signal provided from network interface 470.

The NPU 458 is dedicated hardware for an artificial neural network, and may include a plurality of computational nodes corresponding to at least some artificial neurons constituting the artificial neural network, and at least some of the plurality of computational nodes are signaled in parallel. Can handle. Since the quantized artificial neural network according to an embodiment of the present disclosure has not only high accuracy but also low computational complexity, it can be easily implemented in the electronic device 400 of FIG. And it can be implemented by a small scale NPU (458).

The input/output devices 430 may include input devices such as a touch input device, a sound input device, and a camera, and output devices such as a display device and a sound output device. For example, when a user's voice is input through a sound input device, the voice may be recognized by an artificial neural network implemented in the electronic device 400, and an operation according to the voice may be triggered. In addition, when an image is input through a camera, an object included in the image may be recognized by a deep neural network implemented in the electronic device 400, and an output such as virtual reality may be provided to the user. . The network interface 470 may provide the electronic device 400 with access to a mobile communication network such as Long Term Evolution (LTE) or 5G, or may provide access to a local network such as Wi-Fi.

As described above, exemplary embodiments have been disclosed in the drawings and specification. In the present specification, embodiments have been described using specific terms, but these are only used for the purpose of describing the technical idea of the present disclosure, and are not used to limit the meaning or the scope of the present disclosure described in the claims. . Therefore, those of ordinary skill in the art will understand that various modifications and equivalent other embodiments are possible therefrom. Therefore, the true technical scope of the present disclosure should be determined by the technical spirit of the appended claims.

Claims (20)

In the computing system,
A neural network system that drives an artificial neural network; And
Including; a quantization system for quantizing the artificial neural network,
The quantization system,
By quantizing the parameters of the artificial neural network, quantized parameters of the artificial neural network are generated, quantization errors of the parameters of the artificial neural network are generated based on the parameters of the artificial neural network and the quantized parameters, and the quantized parameter And generating a correction bias based on quantization errors of parameters of the artificial neural network, and transmitting the generated quantized parameters and the correction bias to the neural network system. The method of claim 1,
The quantization system,
Upon receiving an input sample from the neural network system, quantize the input sample, transmit the quantized input sample to the neural network system,
The neural network system,
Upon receiving the quantized input sample, a first multiply-accumulate (MAC) operation is performed based on the quantized input sample and the quantized parameters, and the correction bias is reflected in the result of the first MAC operation. A computing system that generates computational results. The method of claim 1,
The parameters of the artificial neural network are,
Including the weight and bias of the artificial neural network,
The quantized parameters are,
Including quantized weights and quantized biases,
The quantization error of the parameters is,
A computing system including a quantization error of the weight and a quantization error of the bias. The method of claim 3,
The quantization system,
Identify a reference sample to generate the correction bias, generate a quantized reference sample by quantizing the reference sample, generate a quantization error of the reference sample based on the reference sample and the quantized reference sample, and the A computing system for generating the correction bias based on at least one of a reference sample, the quantized reference sample, a quantization error of the reference sample, the quantized parameters, and a quantization error of parameters of the artificial neural network. The method of claim 4,
The quantization system,
A second MAC operation is performed based on a quantization error of the reference sample and the weight, a third MAC operation is performed based on the quantization error of the reference sample and the quantized weight, and a result of the second MAC operation and A computing system for generating the correction bias based on a result of the third MAC operation. The method of claim 5,
The quantization system,
A computing system for generating the correction bias by summing a result of the second MAC operation, a result of the third MAC operation, and a bias of the artificial neural network. The method of claim 6,
The quantization system,
A computing system that calculates an average value of the summation result and generates the correction bias having the calculated average value as a scalar value. The method of claim 4,
The quantization system,
A fourth MAC operation is performed based on the quantization error of the quantized reference sample and the weight, a fifth MAC operation is performed based on the quantization error of the reference sample and the weight of the artificial neural network, and the fourth MAC operation The computing system for generating the correction bias based on the result of the fifth MAC operation and the result of. The method of claim 4,
The neural network system,
Select at least one first sample from a sample pool, and transmit the at least one first sample to the quantization system,
The quantization system,
Generating the reference sample based on the at least one first sample,
A computing system for generating at least one quantized first sample by quantizing the at least one first sample, and generating the quantized reference sample using the at least one quantized first sample. The method of claim 9,
The neural network system,
Computing system for selecting a plurality of second samples from the sample pool, performing an operation on the plurality of second samples through the artificial neural network, and selecting the at least one first sample based on the calculation result . The method of claim 10,
The neural network system,
A computing system that checks statistical distributions of output samples of each of the layers constituting the artificial neural network based on the calculation result, and selects the at least one first sample based on the checked statistical distributions. The method of claim 3,
The quantization system,
An expected input sample is generated by predicting a next sequence of samples based on at least one third sample processed by the neural network system based on the quantized parameters, and based on a quantization error of the at least one third sample A computing system for generating a quantization error of the expected input sample, and generating the correction bias based on a quantization error of the expected input sample, a quantization error of the expected input sample, the quantized parameters, and parameters of the artificial neural network. The method of claim 12,
The quantized system,
A computing system that calculates a motion vector of the at least one third sample and generates the expected input sample using the calculated motion vector. The method of claim 12,
The quantized system,
A computing system for generating the expected input sample by inputting the at least one third sample into an artificial intelligence module that is learned to predict a next sequence of samples based on a previous sequence of samples. In the computation method using an artificial neural network,
Quantizing weights and biases of the artificial neural network;
Generating a correction bias by correcting the quantized bias to include an error due to quantization;
Quantizing the input samples;
Performing a first multiply-accumulate (MAC) operation based on the quantized weight of the artificial neural network and the quantized input sample; And
And reflecting the correction bias to a result of the first MAC operation. The method of claim 15,
Generating the correction bias,
A calculation method for correcting the quantized bias based on a first error that is a quantization error of a reference sample and a second error that is a quantization error of the weight. The method of claim 16,
Generating the correction bias,
Selecting at least one first sample from a sample pool and generating a reference sample based on the at least one first sample; And
Calculating a quantization error of the reference sample based on the quantization error of the at least one first sample. The method of claim 17,
The step of calculating the reference sample,
Predicting the input sample based on at least one first sample already processed through the artificial neural network among the sample pool; And
Generating the expected input sample as the reference sample. In the quantization method of an artificial neural network,
Quantizing parameters of the artificial neural network;
Calculating a quantization error of the parameters based on the parameters of the artificial neural network and the quantized parameters;
Generating a correction bias based on the quantized parameters and quantization errors of the parameters. The method of claim 19,
Generating the correction bias,
Generating a reference sample based on at least one first sample;
Quantizing the reference sample;
Calculating a quantization error of the reference sample based on the reference sample and the quantized reference sample; And
Generating the correction bias using at least one of the reference sample, the quantized reference sample, a quantization error of the reference sample, the quantized parameters, and a quantization error of the parameters.

Images