KR102368590B1 - Electronic apparatus and control method thereof - Google Patents

Abstract

An electronic device is disclosed. The electronic device includes a memory and a processor in which an artificial intelligence model composed of a plurality of layers is stored, and the artificial intelligence model is scaled based on a different shift scaling factor for a plurality of channels included in each of the plurality of layers, and is quantized for each of the plurality of layers. and a plurality of weighted values, and when input data is received, the processor calculates an operation result for each channel in a neural network operation process on the input data, based on a shift scaling factor corresponding to each channel, and calculates an inversely scaled synthetic scale parameter can do.

Description

Electronic device and its control method { ELECTRONIC APPARATUS AND CONTROL METHOD THEREOF }

The present disclosure relates to an electronic device and a control method thereof, and more particularly, processing an artificial intelligence (AI) model that mimics functions such as cognition and judgment of the human brain by using a machine learning algorithm such as deep learning It relates to an electronic device and a method for controlling the same.

Recently, quantization has been used to increase the compression rate while minimizing the performance degradation of deep learning models. The weight quantization method can be divided into post-training quantization and quantization-aware training based on the time of quantization, and linear quantization based on the quantization method. and non-linear quantization.

Quantization after learning uses an already trained Float32 model to perform IntN quantization without re-learning, so the quantization speed is fast and training data is not required. Linear quantization is universally supported in Neural Processing Units (NPUs) as it can be easily implemented in hardware with INT MULTIPLYER and INT ADDER.

In spite of these advantages, in general, the combination of quantization and linear quantization after learning has a disadvantage in that the accuracy loss becomes larger as the condition of small quantum bits (8 bits or less) after quantization is compared to the combination of quantization and nonlinear quantization during learning. This is because a plurality of channels included in one layer have different parameter distributions, and in particular, a channel having a small parameter distribution converges to one quantum value after quantization, resulting in a large quantum error.

Various methods have been developed to overcome these shortcomings.

First, in channel-wise quantization, quantization is performed in units of neural network layers as in the existing method to obtain a pair of minimum and maximum parameters for each layer, and from this, quantum parameters (QWeight, Scale) required for hardware fixed computing are obtained. , zero point), quantization is performed in units of channels included in the layer to obtain a pair of minimum and maximum parameters for each channel. For example, in the case of n channels, n [min, max] may be obtained.

Although the loss of quantization precision is reduced through this operation, there is a problem in that the size of a quantum parameter increases in proportion to the number of channels. This increases the time to load the quantum parameters from the main memory to the cache memory, which degrades the latency performance.

(float) scaling,

(float) rescaling를 적용하는 전처리를 수행한다. 신경망 첫 레이어부터 마지막 레이어까지 연속적으로 스케일을 조정하며, 이를

의 변화가 없게되는 때까지 전체를 다시 반복 수행하며, 전처리가 종료되면 일반적인 Layer-wise quantization를 수행한다.Cross Layer Equalization (CLE) is performed on the output of the front convolution layer and the corresponding input of the back convolution layer.

(float) scaling,

(float) Performs preprocessing that applies rescaling. It scales continuously from the first layer to the last layer of the neural network, and

The whole process is repeated again until there is no change in , and when the pre-processing is finished, general layer-wise quantization is performed.

Through this preprocessing, the range of parameters for each channel is adjusted to overlap, so the loss of quantization precision is small. there is. Accordingly, it is necessary to replace ReLU6 or PReLU with ReLU, and accordingly, the activated feature map distribution changes, the error increases, and the accuracy decreases.

Accordingly, there is a need to develop a method that does not require structural change of the neural network layer, is advantageous for hardware operation, and increases quantization efficiency while maintaining quantization accuracy.

The present disclosure has been made in accordance with the above-mentioned necessity, and an object of the present disclosure is to provide an electronic device for performing a neural network operation using an artificial intelligence model shift-scaled for each channel and quantized for each layer, and a method for controlling the same.

In order to achieve the above object, an electronic device according to an embodiment of the present disclosure includes a memory and a processor in which an artificial intelligence model composed of a plurality of layers is stored, and the artificial intelligence model includes a plurality of layers included in each of the plurality of layers. includes a plurality of weight values scaled based on a different shift scaling factor for each channel and quantized for each of the plurality of layers of may be calculated with the inverse-scaled synthetic scale parameter based on the shift scaling factor corresponding to each channel.

In addition, the artificial intelligence model includes the plurality of quantized weight values, a shift scaling factor for each channel included in each of the plurality of layers, and a scale parameter and a zero point parameter corresponding to each of the plurality of layers, The scale parameter represents a gradient between a value before quantization and a value after quantization when each of the plurality of layers is quantized, and the zero point parameter represents a value after quantization of a zero value before quantization when quantizing each of the plurality of layers. can

In addition, the processor inversely scales a value obtained based on a scale parameter of a current layer, a scale parameter of a layer immediately before the current layer, and a scale parameter of the plurality of weight values based on a shift scaling factor corresponding to each channel. to obtain the synthesis scale parameter.

In addition, the processor may obtain the synthesized scale parameter by shifting the obtained value based on a shift scaling factor corresponding to each channel.

Here, the obtained value may be inversely proportional to a scale parameter of the current layer, and may be proportional to a scale parameter of a layer immediately before the current layer and a scale parameter of the plurality of weight values.

In addition, when it is identified that the shift scaling factors for each channel included in each of the plurality of layers are included in the artificial intelligence model, the processor may perform the inverse scaling.

Also, the electronic device may be implemented as a Neural Processing Unit (NPU).

The shift scaling factor corresponding to each channel may be determined based on a weight value included in each channel and a weight value included in a layer including each channel.

In addition, the shift scaling factor corresponding to each channel may be determined based on a weight value having a largest magnitude in each channel and a weight value having a largest magnitude in a layer including each channel.

Meanwhile, in the method of controlling an electronic device according to an embodiment of the present disclosure, each of a plurality of layers constituting the artificial intelligence model in the step of receiving input data and a neural network calculation process for the input data using the artificial intelligence model calculating an operation result for each channel with an inversely scaled synthetic scale parameter based on a shift scaling factor corresponding to each channel, wherein the artificial intelligence model performs a different shift for each channel included in each of the plurality of layers. A plurality of weight values scaled based on a scaling factor and quantized for each of the plurality of layers may be included.

In addition, the artificial intelligence model includes the plurality of quantized weight values, a shift scaling factor for each channel included in each of the plurality of layers, and a scale parameter and a zero point parameter corresponding to each of the plurality of layers, The scale parameter represents a gradient between a value before quantization and a value after quantization when each of the plurality of layers is quantized, and the zero point parameter represents a value after quantization of a zero value before quantization when quantizing each of the plurality of layers. can

In addition, the calculating may include calculating a value obtained based on a scale parameter of a current layer, a scale parameter of a layer immediately before the current layer, and a scale parameter of the plurality of weight values, based on a shift scaling factor corresponding to each channel. The synthesis scale parameter may be obtained by inverse scaling.

In addition, the calculating may include obtaining the synthesized scale parameter by shifting the obtained value based on a shift scaling factor corresponding to each channel.

Here, the obtained value may be inversely proportional to a scale parameter of the current layer, and may be proportional to a scale parameter of a layer immediately before the current layer and a scale parameter of the plurality of weight values.

In addition, in the calculating, when it is identified that the shift scaling factors for each channel included in each of the plurality of layers are included in the artificial intelligence model, the calculation may be performed.

Also, the electronic device may be implemented as a Neural Processing Unit (NPU).

The shift scaling factor corresponding to each channel may be determined based on a weight value included in each channel and a weight value included in a layer including each channel.

In addition, the shift scaling factor corresponding to each channel may be determined based on a weight value having a largest magnitude in each channel and a weight value having a largest magnitude in a layer including each channel.

According to various embodiments of the present disclosure as described above, the electronic device calculates the operation result for each channel with the inversely scaled synthetic scale parameter based on the shift scaling factor corresponding to each channel in the neural network operation process for the input data, so that the relative It is possible to improve the accuracy of neural network computation while using an artificial intelligence model implemented with a small capacity.

1A is a block diagram illustrating a configuration of an electronic device according to an embodiment of the present disclosure.
1B is a block diagram illustrating a software configuration of an electronic device according to an embodiment of the present disclosure.
1C is a diagram for explaining scaling for each channel according to an embodiment of the present disclosure.
1D and 1E are diagrams for explaining a method of implementing a shifter according to various embodiments of the present disclosure.
2 is a diagram for explaining operations of a compiler and an electronic device according to an embodiment of the present disclosure.
3 is a flowchart illustrating a method of obtaining a shift scaling factor according to various embodiments of the present disclosure.
4A and 4B are diagrams for explaining an inverse scaling operation according to an embodiment of the present disclosure.
5A to 5C are diagrams for explaining the effect according to the present disclosure.
6 is a flowchart illustrating a method of controlling an electronic device according to an embodiment of the present disclosure.

Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

Terms used in the embodiments of the present disclosure are selected as currently widely used general terms as possible while considering the functions in the present disclosure, which may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, etc. . In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding disclosure. Therefore, the terms used in the present disclosure should be defined based on the meaning of the term and the contents of the present disclosure, rather than the simple name of the term.

In this specification, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of a corresponding characteristic (eg, a numerical value, function, operation, or component such as a part). and does not exclude the presence of additional features.

The expression "at least one of A and/or B" is to be understood as indicating either "A" or "B" or "A and B".

As used herein, expressions such as "first," "second," "first," or "second," can modify various elements, regardless of order and/or importance, and refer to one element. It is used only to distinguish it from other components, and does not limit the components.

The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as "comprises" or "consisting of" are intended to designate that the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification exist, and are intended to indicate that one or more other It should be understood that this does not preclude the possibility of addition or presence of features or numbers, steps, operations, components, parts, or combinations thereof.

In this specification, the term user may refer to a person who uses an electronic device or a device (eg, an artificial intelligence electronic device) using the electronic device.

Hereinafter, various embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings.

1A is a block diagram illustrating a configuration of an electronic device 100 according to an embodiment of the present disclosure. The electronic device 100 includes a memory 110 and a processor 120 as shown in FIG. 1A .

The electronic device 100 may be a device that performs a neural network operation based on an artificial intelligence model. For example, the electronic device 100 stores an artificial intelligence model, and when input data is received, a device that performs a neural network operation on the input data based on the artificial intelligence model, and is implemented with a desktop PC, a laptop computer, a TV, etc. can be However, the present invention is not limited thereto, and the electronic device 100 may be any device as long as it is capable of performing a neural network operation based on an artificial intelligence model.

In particular, the electronic device 100 is a device with limited resources, such as a smart phone, a tablet PC, a wearable device, etc., and may be a device that stores a quantized artificial intelligence model and performs a neural network operation based on the quantized artificial intelligence model. . Quantization means dividing a continuous value into a plurality of levels, and substituting a value within each level with a value representing each level.

For example, a data size may be reduced through quantization in which a value between 0 and 1 is replaced with 1 and a value between 1 and 2 is replaced with 2. That is, the neural network operation may be performed in an on-device form even in the electronic device 100 having limited resources by quantizing the artificial intelligence model. A detailed description of the quantization of the artificial intelligence model will be described later.

The memory 110 may refer to hardware that stores information such as data in an electrical or magnetic form so that the processor 120 can access it. To this end, the memory 110 may be implemented as hardware at least one of non-volatile memory, volatile memory, flash memory, hard disk drive (HDD) or solid state drive (SSD), RAM, ROM, etc. .

At least one instruction or module required for the operation of the electronic device 100 or the processor 120 may be stored in the memory 110 . Here, the instruction is a code unit for instructing the operation of the electronic device 100 or the processor 120 , and may be written in machine language, which is a language that a computer can understand. A module may be a set of instructions that perform a specific task of a unit of work.

The memory 110 may store data that is information in units of bits or bytes that can represent characters, numbers, images, and the like. For example, data such as a document including a plurality of sentences may be stored in the memory 110 .

An artificial intelligence model composed of a plurality of layers may be stored in the memory 110 . Here, the artificial intelligence model may include a plurality of weight values scaled based on a different shift scaling factor for each of a plurality of channels included in each of the plurality of layers and quantized for each of the plurality of layers.

For example, if it is assumed that the AI model before scaling and quantization consists of 5 layers, and each of the 5 layers contains 32 channels, the AI model before scaling and quantization may include a total of 160 channels. . First, each of the 160 channels may be scaled based on a different shift scaling factor. The upper part of FIG. 1C shows a plurality of channels included in one of five layers included in the artificial intelligence model. The lower part of FIG. 1C is a diagram in which each of a plurality of channels is scaled by a different shift scaling factor.

In this way, the shift scaling factors may be different for each channel, and accordingly, the total number of shift scaling factors may be 160. Here, the shift scaling factor for each channel may be determined in a power-of-two form to be performed by a shift operation in hardware. For example, the first channel may be scaled based on shift by 3, and the second channel may be scaled based on shift by 5. In this case, the shift scaling factor applied to each channel may be determined based on a weight value having the largest magnitude in each channel and a weight value having the largest magnitude in a layer including each channel. For example, if the weight value with the largest size in the first layer among the five layers is 10 and the weight value with the largest size in the first channel included in the first layer is 6, the logarithmic ratio between 10 and 6 (logarithmic) ratio), an initial value of the shift scaling factor of the first channel included in the first layer may be determined. And, the quantum-inverse quantization error value or Top-1 test accuracy is defined as a cost function, and the optimal value of the shift scaling factor for each channel can be obtained through a nonlinear optimization method (Nelder-Mead, Bayesian Optimization, etc.). . As described above, a channel with a relatively small deviation may have a relatively large shift scaling factor, and a channel with a relatively large deviation may have a relatively small shift scaling factor. can be secured.

When scaling is completed, quantization is performed for each layer. In the above-described example, for example, if the minimum and maximum values of the weight values of the first layer are mapped to 0 and 255, respectively, the weight value of the first layer may be replaced with an integer between 0 and 255.

As described above, the artificial intelligence model shift-scaled for each channel and quantized for each layer has a plurality of quantized weight values, a shift scaling factor for each channel included in each of the plurality of layers, and a scale parameter and zero point corresponding to each of the plurality of layers. It may contain parameters. Here, the scale parameter represents a gradient between a value before quantization and a value after quantization when each of a plurality of layers is quantized, and a zero point parameter represents a value after quantization of a zero value before quantization when each of a plurality of layers is quantized. there is.

This requires additional data as much as a shift scaling factor for each channel than when performing layer-by-layer quantization without scaling for each channel in the prior art. According to the present disclosure, there is an advantage in solving this problem through shift scaling for each channel. Alternatively, in the case of performing quantization for each channel in the related art, accuracy is secured, but quantization parameters (38-bit floating-point scale and 8-bit integer zero point) are required for each channel, which requires considerable data. , according to the present disclosure, while securing a certain level of accuracy through shift scaling for each channel, there is an advantage that the added data is only a shift scaling factor for each channel in the form of a 4-bit integer.

That is, the artificial intelligence model quantized for each layer after shift scaling for each channel requires an additional storage capacity as much as the shift scaling factor for each channel compared to the case where quantization is performed for each layer without conventional scaling, but is insignificant compared to the capacity of the weight, which is on-device There is no big problem in performing neural network calculations in the form. Nevertheless, if a quantized artificial intelligence model is used for each layer after scaling for each channel, a certain level of accuracy can be secured, and this will be described along with the operation of the processor 120 .

The processor 120 controls the overall operation of the electronic device 100 . Specifically, the processor 120 may be connected to each component of the electronic device 100 to control the overall operation of the electronic device 100 . For example, the processor 120 may be connected to components such as the memory 110 and a communication interface (not shown) to control the operation of the electronic device 100 .

According to an embodiment, the processor 120 may be implemented as a digital signal processor (DSP), a microprocessor, or a time controller (TCON). However, the present invention is not limited thereto, and the central processing unit ( central processing unit (CPU)), micro controller unit (MCU), micro processing unit (MPU), controller, application processor (AP), or communication processor (CP), ARM processor In addition, the processor 120 may be implemented as a SoC (System on Chip) or LSI (large scale integration) in which a processing algorithm is embedded, or an FPGA ( Field programmable gate array) may be implemented.

When input data is received, the processor 120 may calculate an operation result for each channel in a neural network operation process on the input data with a synthetic scale parameter descaled based on a shift scaling factor corresponding to each channel. As described above, this operation is intended to restore the scale of the output, which is different according to the shift scaling for each channel, to the original scale in order to reduce the quantum error.

The operation of the processor 120 will be described in more detail through various modules of FIG. 1B .

1B is a block diagram illustrating a software configuration of the electronic device 100 according to an embodiment of the present disclosure. Positioning a plurality of modules inside the processor 120 in FIG. 1b is to indicate a state in which the plurality of modules are loaded (or executed) by the processor 120 and operated in the processor 120, and the plurality of modules are It may be in a state previously stored in the memory 110 .

Referring to FIG. 1B , an artificial intelligence model and input data shift-scaled for each channel and quantized for each layer may be stored in the memory 110 . Here, the artificial intelligence model may include a scale parameter and a zero point parameter.

In addition, the processor 120 may control the overall operation of the electronic device 100 by executing a module or an instruction stored in the memory 110 . Specifically, the processor 120 reads and interprets the module or instruction, and may determine a sequence for data processing, and accordingly controls the operation of the other configuration by transmitting a control signal that controls the operation of the other configuration, such as the memory 110 . can do.

The processor 120 may apply the input data to the quantized artificial intelligence model by executing the neural network operation module and the inverse scaling module for each channel. In this case, the processor 120 may perform a neural network operation on the input data, and may obtain a synthetic scale parameter for inverse scaling of an operation result for each channel. Here, the neural network operation module and the inverse scaling module for each channel may be physically implemented as one module or may be implemented in a separate form.

For example, the processor 120 may calculate the input data or the feature map data with the weight value of the corresponding channel, and then calculate the operation result with the inverse-scaled synthetic scale parameter based on the corresponding shift scaling factor. Specifically, the processor 120 calculates the input data with a weight value of each of the plurality of first channels included in the first layer, and calculates the calculation result with the weight value of each of the plurality of first channels, respectively, of the plurality of first channels. An operation may be performed with the inverse-scaled synthetic scale parameter based on the shift scaling factor corresponding to . Then, the processor 120 calculates the feature map data output from the first rater with a weight value of each of a plurality of second channels included in a second layer after the first layer, and a weight value of each of the plurality of second channels A result of the operation of and may be calculated with the inverse-scaled synthetic scale parameter based on the shift scaling factor corresponding to each of the plurality of second channels. The inverse scaling operation of the processor 120 is implemented as a shift operation, and the shift operation and the composite scale parameter will be described later.

First, the weight value included in the AI model is a value obtained by being shift-scaled based on a different shift scaling factor for each channel included in the AI model and quantized for each layer. To explain this, we describe the structure of the data of the artificial intelligence model.

The artificial intelligence model may include a scale parameter and a zero point parameter corresponding to each of the plurality of layers. Here, the scale parameter represents a gradient between a value before quantization and a value after quantization when each of a plurality of layers is quantized, and a zero point parameter represents a value after quantization of a zero value before quantization when each of a plurality of layers is quantized. there is.

For example, if quantization is performed in such a way that the minimum and maximum values among the weight values of the first layer are mapped to 0 and 255, respectively, a scale parameter indicating the correlation between the minimum and maximum values of the weight values and 0 and 255, zero Point parameters and quantized weight values may be obtained. Here, the scale parameter refers to a slope indicating the correlation between data before and after quantization, and the zero point parameter refers to a quantum value representing the real number 0.0 or the degree to which the correlation deviates from the origin.

Through the method described above, a scale parameter and a zero point parameter for a weight for each layer may be obtained. In addition, scale parameters and zero point parameters for input and output for each layer may be obtained by the same method. The processor 120 inversely scales a value obtained based on a scale parameter of the current layer, a scale parameter of a layer immediately before the current layer, and a scale parameter of the plurality of weight values based on a shift scaling factor corresponding to each channel. to obtain the synthesis scale parameter.

For example, the processor 120 may obtain an output value quantized by Equation 1 below. The acquisition process of Equation 1 will be described later with reference to the drawings.

[Equation 1]

는 각각 이전 레이어의 출력 데이터로서 피쳐 맵 데이터(또는, 입력 데이터), 이전 레이어의 스케일 파라미터, 이전 레이어의 제로 포인트 파라미터이고,

는 각각 양자화된 가중치 값, 가중치 값의 스케일 파라미터, 가중치 값의 제로 포인트 파라미터이고,

는 각각 현재 레이어의 출력 데이터, 현재 레이어의 스케일 파라미터, 현재 레이어의 제로 포인트 파라미터이다.

는 floating bias

를

스케일로 symmetric quantization 한 후의 양자 bias 값이고, i, j는 각각 출력 채널 인덱스, 입력 채널 인덱스이다.here,

are the feature map data (or input data) as output data of the previous layer, the scale parameter of the previous layer, and the zero point parameter of the previous layer, respectively,

are a quantized weight value, a scale parameter of a weight value, and a zero point parameter of a weight value, respectively;

are output data of the current layer, a scale parameter of the current layer, and a zero point parameter of the current layer, respectively.

is a floating bias

cast

It is a quantum bias value after symmetric quantization with scale, and i and j are output channel indexes and input channel indexes, respectively.

The processor 120 obtains a weight value, a zero point parameter (previous layer, a current layer, a weight value), and input data from the memory 110 by executing the neural network operation module, and executes the inverse scaling module for each channel to the memory 110 ), it is possible to obtain a shift scaling factor for each channel and a scale parameter (previous layer, current layer, weight value). The inverse scaling module for each channel inversely scales a value obtained based on the scale parameter of the previous layer, the scale parameter of the current layer, and the scale parameter of the weight value, based on the shift scaling factor corresponding to each channel to obtain a composite scale parameter can do. That is, the process of obtaining the synthetic scale parameter includes inverse scaling, and the inverse scaling module for each channel may provide the synthetic scale parameter to the neural network operation module. Here, the obtained value may be inversely proportional to the scale parameter of the current layer, and may be proportional to the scale parameter of the layer immediately before the current layer and the scale parameter of the plurality of weight values.

를

의 형태로 변환하고, 각 채널에 대응되는 시프트 스케일링 팩터(

)를

와 같이 추가함에 따라 역스케일링된 합성 스케일 파라미터를 획득할 수 있다.That is, the processor 120 is an operation value between the scale parameters of Equation 1,

cast

converted to the form of , and the shift scaling factor (

)cast

By adding as , an inversely scaled synthetic scale parameter can be obtained.

In particular, since the processor 120 processes data in a binary format, addition of a shift scaling factor may be implemented as a shift operation. That is, the processor 120 may obtain the synthesized scale parameter by shifting the obtained value based on the shift scaling factor corresponding to each channel, and it is easy to implement hardware because it simply adds a shift operation.

The processor 120 may perform an operation as in Equation 1 by executing the neural network operation module. Specifically, the processor 120 may include a weight value, a zero point parameter (previous layer, a current layer, a weight value), and input data. , a neural network operation may be performed based on the synthetic scale parameter. Here, the zero point parameter of the previous layer and the current layer is a fixed value during the calculation of the current layer, and the zero point parameter of the weight value is the same as a value used for quantization of the weight value until the layer is changed. In addition, the synthesized scale parameter obtained through the inverse scaling module for each channel in the process of performing the neural network operation may be different for each channel.

를 신경망 연산에 이용하고, 인공 지능 모델이 채널 별 시프트 스케일링 팩터를 포함하지 않는 경우

를 신경망 연산에 이용하게 된다.Meanwhile, when it is identified that the quantized artificial intelligence model includes a plurality of shift scaling factors for each channel included in each of the plurality of layers by executing the inverse scaling module for each channel, the processor 120 may perform inverse scaling. That is, the processor 120 may determine whether to perform the inverse scaling operation based on whether the artificial intelligence model includes the shift scaling factor for each channel. Specifically, the processor 120 performs inverse scaling through a shift operation when the artificial intelligence model includes the shift scaling factor for each channel, and does not perform the shift operation when the artificial intelligence model does not include the shift scaling factor for each channel. it may not be That is, when the artificial intelligence model includes the shift scaling factor for each channel, the processor 120

is used for neural network computation, and the artificial intelligence model does not include the shift scaling factor for each channel.

is used for neural network computation.

Meanwhile, the electronic device 100 may be implemented as a Neural Processing Unit (NPU). In this case, a cache memory included in the neural network processing apparatus may operate as the memory 110 , and a plurality of processing elements included in the neural network processing apparatus may operate as the processor 120 .

As described above, the processor 120 can perform the inverse scaling operation in the neural network calculation process, and can obtain the result of the inverse scaling only by shifting some data, making it easy to implement in an on-device form.

In addition, in the quantization process of the artificial intelligence model, there is an effect that the data of each channel is not crushed according to the scaling of each channel, so that a certain level of accuracy can be secured.

Meanwhile, as described above, the inverse scaling operation may be implemented through a shifter. For example, as shown in FIG. 1D , the shifter may be implemented as a component inside the processor 120 . That is, the inverse scaling module for each channel of FIG. 1B may be implemented as a shifter.

Alternatively, the shifter 130 may be implemented as an external component of the processor 120 as shown in FIG. 1E . In this case, the shifter 130 receives the scale parameter and the shift scaling factor for each channel from the memory 110, and a value obtained based on the scale parameter of the previous layer, the scale parameter of the current layer, and the scale parameter of the weight value, A synthesized scale parameter may be obtained by performing inverse scaling based on a shift scaling factor corresponding to each channel. In addition, the shifter 130 may provide the synthesized scale parameter to the processor 120 .

When a shifter is implemented as shown in FIG. 1E, even if a conventional processor is used, reverse scaling as in the present disclosure is possible.

Hereinafter, various embodiments of the present disclosure will be described in more detail with reference to the drawings.

2 is a diagram for explaining the operations of the compiler 200 and the electronic device 100 according to an embodiment of the present disclosure.

The compiler 200 may scale a plurality of weight values included in an input model (Float Model File) based on different shift scaling factors for a plurality of channels included in each of a plurality of layers, and may quantize the values for each of the plurality of layers.

Compiler 200 includes a parsing module that parses an input model, an instruction stream module that reconfigures an op supported by a custom NPU, a weight equalizing scaler (WES) module, a quantizer module that quantizes Float parameters into IntN, memory distribution, It can include an optimization module that optimizes (tiles) an operation and a Binarization module that makes it into a binary file.

In particular, the WES module can receive from the Instruction Stream module a graph file composed of NPU HW Operation, connected like a chain in one direction, and parameters in Float32 format.

The WES module acquires the minimum and maximum values of the original parameter for each channel as shown in the upper part of FIG. 1C, and sets a reference range to have the minimum quantization error using the minimum and maximum values for each channel, thereby setting the shift scaling factor for each channel (channel- wise shift scale value) can be obtained.

Specifically, the WES module obtains the range of the original parameter for each channel, sets the range having the maximum value (channel 22 in the upper part of FIG. 1C ) as the reference range, and obtains the shift scaling factor for each channel based on the reference range can do.

For example, the WES module may identify a weight value having the largest size among weight values included in one layer, and obtain a shift scaling factor of each channel based on the identified weight value. In this case, the WES module may obtain the shift scaling factor of each channel based on the weight value having the largest magnitude in each channel and the weight value identified above.

Alternatively, the WES module may obtain the shift scaling factor of each channel by applying the gradient-descent method so that the ratio of the changed range values is added up after scaling each channel for the entire range. .

Alternatively, the WES module shift scales each channel, performs quantization to convert to int, and inverse quantization process to recover floating, summing up quantum errors for each channel so that the value has a minimum value. A shift scaling factor of the channel may be obtained.

The WES module can obtain Si by dividing the reference range by the range of each channel, then taking the algorithm with exponent 2 and taking its rounded down value. 2^Si can be easily processed by bit shift (<< Si) operation in hardware that calculates binary numbers, so hardware implementation is easy.

As shown at the bottom of FIG. 1c, the parameters wi updated according to the 2^Si type scaling for each channel match the minimum and maximum values of the entire layer to the maximum, so that the optimal range for layer-wise quantization is obtained. can have In this case, the same 2^Si scale may be applied to bias(bi).

Thereafter, layer-wise linear quantization may be performed on the convolutional layer whose quantization module parameter scale is adjusted.

Meanwhile, the electronic device (NPU HW, 100) may include an ALU module in charge of fixed computing, a cache memory that stores parameters and input/output feature maps required for operation in each cycle, and a memory that shares all parameters and feature maps. .

Here, according to the operation of the WES module of the compiler 200, the ALU module may be structurally changed to Fixed Computing ALU w/ Ch-wise Shift Scaling, which will be described in more detail with reference to the following drawings.

3 is a flowchart illustrating a method of obtaining a shift scaling factor according to various embodiments of the present disclosure.

First, the WES module may obtain the range of the original parameter for each channel (S310). Then, the WES module selects the entire range (S320), obtains a shift scaling factor for each channel based on the entire range and the range for each channel, and performs shift scaling for each channel based on the obtained shift scaling factor for each channel. There is (S330).

After shift scaling for each channel, the WES module additionally performs quantization for each layer (S331), calculates a quantization error (S332), re-acquires a shift scaling factor for each channel based on the quantization error, and re-acquired shift for each channel Shift scaling for each channel may be re-performed based on the scaling factor (S390).

Alternatively, the WES module additionally defines a cost function for the entire range after shift scaling for each channel (S340), performs shift scaling for each channel based on the shift scaling factor for each channel (S350), and quantizes after quantization (S360) An error may be calculated (S370). When the quantization error converges to a preset value, the WES module may determine a shift scaling factor for each channel and perform scaling for each channel based on the determined shift scaling factor for each channel ( S390 ). Alternatively, when the quantization error does not converge to a preset value, the WES module sums the ratio of the range values changed after scaling, and applies the gradient-descent method so that the value increases, and the shift scaling factor for each channel can be readjusted. .

In the above manner, the WES module may obtain a shift scaling factor for each channel.

4A and 4B are diagrams for explaining an inverse scaling operation according to an embodiment of the present disclosure.

As shown in FIG. 4A , the convolution operation and the scaler on the left side may be implemented as one configuration as shown on the right side. First, the configuration on the left will be described.

Fixed computing ALU performing INT operation can load input data of INTN quantized values, weight values, scale parameters, and zero point parameters from memory. Here, the scale parameter and the zero point parameter may include a scale parameter of the current layer, a scale parameter of a layer immediately before the current layer, and a scale parameter of a plurality of weight values.

The scale parameter refers to a slope indicating the correlation between data before and after quantization, and the zero point parameter refers to a quantum value representing the real number 0.0 or the degree to which the correlation deviates from the origin. For example, as shown in FIG. 4B , the maximum value (max) of the real axis may be quantized to 255, and the minimum value (min) of the real axis may be quantized to 0, and the slope at this time is a scale parameter. And, the zero value of the real axis is quantized to z, where z is the zero point parameter.

In this way, a scale parameter and a zero point parameter for each layer may be obtained.

,

,

은 각각 이전 레이어, 현재 레이어 및 가중치 값에 대한 int to float 변환하는 float 스케일 값이고, 레이어마다 상이할 수 있다. Fixed computing ALU는

를

(M : mantissa or multiplier,

: exponent or shiftamount)의 형태로 변환할 수 있다.The fixed computing ALU may obtain an output value from an input value and a parameter quantized in the Fixed Computing Convolution layer as shown in Equation 1 above. here,

,

,

is a float scale value that converts int to float for the previous layer, the current layer, and the weight value, respectively, and may be different for each layer. Fixed computing ALU is

cast

(M: mantissa or multiplier,

: exponent or shiftamount).

)를

와 같이 추가함에 따라

가 역스케일링된 합성 스케일링 파라미터를 획득할 수 있으며, 도 4a의 우측과 같이 역스케일링 동작을 컨볼루션 연산 중에 부가적으로 수행할 수 있다. 특히, 역스케일링 동작은 채널 별 반대 방향으로의 시프트 형태로 처리될 수 있어 하드웨어적인 구현이 용이하다.Fixed computing ALU can additionally load integer Si expressed as 4 bits out of 2^Si for each channel from memory. Integer Si for each channel may be received from the compiler 200 and stored in the memory 110 . And, the fixed computing ALU is a shift scaling factor (

)cast

by adding as

may obtain an inverse-scaled synthetic scaling parameter, and an inverse-scaling operation may be additionally performed during a convolution operation as shown on the right side of FIG. 4A . In particular, since the inverse scaling operation can be processed in the form of a shift in the opposite direction for each channel, hardware implementation is easy.

Meanwhile, it has been described above that the compiler 200 provides information on the shift scaling factor for each channel to the electronic device 100 as an integer Si expressed by 4 bits among 2^Si. For example, in hardware supporting layer-wise quantization, the composite scale is composed of Multiplier(M, 32bit) and Shift amount(S, 6bit) in common for channels. According to the above method, Multiplier(M), Shift amount( S) is used in common for channels, but Ch-wise shift scaling (Si) is received as an additional parameter in a separate 4-bit format from the compiler, and the output value obtained by subtracting the Ch-wise shift scale (Si) from the shift amount (S) Inverse scaling can be performed by scaling to and multiplying by Multiplier. Alternatively, after shifting the output value by the shift amount, shifting by the Ch-wise shift scale again, a double shift operation of multiplying by a multiplier may be performed.

However, the present invention is not limited thereto, and the compiler 200 may provide a shift scaling factor for each channel to which inverse scaling is applied. For example, the Multiplier (M) is used in common channels, but the compiler 200 calculates the shift amount (S) by subtracting the Ch-wise shift scale (Si) from the channel common S for each channel. 6-bit information of the electronic device 100 ) can also be provided.

Meanwhile, although it has been described above that the compiler 200 and the electronic device 100 are separate devices, the two devices may be implemented as one integrated device.

5A to 5C are diagrams for explaining the effect according to the present disclosure.

As shown in FIG. 5A , the accuracy when WES is used is significantly close to that of the baseline without quantization, and exhibits higher accuracy than the prior art.

On the other hand, as shown in FIG. 5B , the size of the parameter when WES is used is considerably smaller than that of quantization for each channel and slightly larger than that of quantization for each layer.

FIG. 5C compares the parameter sizes according to the number of channels, and shows a result similar to that of FIG. 5B. That is, when WES is used, there is a slight increase in parameters compared to the case where quantization is performed for each layer, but the resulting accuracy is close to that in the case of no quantization.

6 is a flowchart illustrating a method of controlling an electronic device according to an embodiment of the present disclosure.

First, input data is received (S610). And, in the neural network calculation process for input data using the artificial intelligence model, the calculation result for each channel of each of the plurality of layers constituting the artificial intelligence model is inversely scaled based on the shift scaling factor corresponding to each channel, and the synthetic scale parameter and Calculate (S620). Here, the artificial intelligence model may include a plurality of weight values scaled based on a different shift scaling factor for each of a plurality of channels included in each of the plurality of layers and quantized for each of the plurality of layers.

In addition, the artificial intelligence model may include a plurality of quantized weight values, a shift scaling factor for each channel included in each of the plurality of layers, and a scale parameter and a zero point parameter corresponding to each of the plurality of layers. Here, the scale parameter represents a gradient between a value before quantization and a value after quantization when each of a plurality of layers is quantized, and a zero point parameter represents a value after quantization of a zero value before quantization when each of a plurality of layers is quantized. there is.

Then, the calculating ( S620 ) is performed by inverting a value obtained based on a scale parameter of the current layer, a scale parameter of a layer immediately before the current layer, and a scale parameter of a plurality of weight values based on a shift scaling factor corresponding to each channel. A synthetic scale parameter may be obtained by scaling.

In addition, in the calculating ( S620 ), the synthesized scale parameter may be obtained by shifting the obtained value based on the shift scaling factor corresponding to each channel.

Here, the obtained value may be inversely proportional to the scale parameter of the current layer, and may be proportional to the scale parameter of the layer immediately before the current layer and the scale parameter of the plurality of weight values.

Then, in the calculating ( S620 ), if it is identified that the shift scaling factors for each channel included in each of the plurality of layers are included in the artificial intelligence model, the calculation may be performed.

Meanwhile, the electronic device may be implemented as a Neural Processing Unit (NPU).

In addition, the shift scaling factor corresponding to each channel may be determined based on a weight value included in each channel and a weight value included in a layer including each channel.

Here, the shift scaling factor corresponding to each channel may be determined based on the largest weight value in each channel and the largest weight value in the layer including each channel.

According to various embodiments of the present disclosure as described above, the electronic device calculates the operation result for each channel with the inversely scaled synthetic scale parameter based on the shift scaling factor corresponding to each channel in the neural network operation process for the input data, so that the relative It is possible to improve the accuracy of neural network computation while using an artificial intelligence model implemented with a small capacity.

Meanwhile, according to a temporary example of the present disclosure, the various embodiments described above may be implemented as software including instructions stored in a machine-readable storage media readable by a machine (eg, a computer). can The device is a device capable of calling a stored command from a storage medium and operating according to the called command, and may include an electronic device (eg, the electronic device A) according to the disclosed embodiments. When the instruction is executed by the processor, the processor may perform a function corresponding to the instruction by using other components directly or under the control of the processor. Instructions may include code generated or executed by a compiler or interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not include a signal and is tangible, and does not distinguish that data is semi-permanently or temporarily stored in the storage medium.

Also, according to an embodiment of the present disclosure, the method according to the various embodiments described above may be included in a computer program product and provided. Computer program products may be traded between sellers and buyers as commodities. The computer program product may be distributed in the form of a machine-readable storage medium (eg, compact disc read only memory (CD-ROM)) or online through an application store (eg, Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

In addition, according to an embodiment of the present disclosure, the various embodiments described above are stored in a recording medium readable by a computer or a similar device using software, hardware, or a combination thereof. can be implemented in In some cases, the embodiments described herein may be implemented by the processor itself. According to the software implementation, embodiments such as procedures and functions described in this specification may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein.

Meanwhile, computer instructions for performing the processing operation of the device according to the above-described various embodiments may be stored in a non-transitory computer-readable medium. When the computer instructions stored in the non-transitory computer-readable medium are executed by the processor of the specific device, the specific device performs the processing operation in the device according to the various embodiments described above. The non-transitory computer-readable medium refers to a medium that stores data semi-permanently, not a medium that stores data for a short moment, such as a register, cache, memory, etc., and can be read by a device. Specific examples of the non-transitory computer-readable medium may include a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In addition, each of the components (eg, a module or a program) according to the above-described various embodiments may be composed of a single or a plurality of entities, and some sub-components of the aforementioned sub-components may be omitted, or other sub-components may be omitted. Components may be further included in various embodiments. Alternatively or additionally, some components (eg, a module or a program) may be integrated into a single entity to perform the same or similar functions performed by each corresponding component prior to integration. According to various embodiments, operations performed by a module, program, or other component are sequentially, parallel, repetitively or heuristically executed, or at least some operations are executed in a different order, are omitted, or other operations are added. can be

In the above, preferred embodiments of the present disclosure have been illustrated and described, but the present disclosure is not limited to the specific embodiments described above, and it is common in the technical field pertaining to the present disclosure without departing from the gist of the present disclosure as claimed in the claims. Various modifications may be made by those having the knowledge of

100: electronic device 110: memory
120: processor 200: compiler

Claims (18)

In an electronic device,
a memory in which an artificial intelligence model composed of a plurality of layers is stored; and
processor; including;
The artificial intelligence model includes a plurality of weight values scaled based on a different shift scaling factor for each of a plurality of channels included in each of the plurality of layers and quantized for each of the plurality of layers,
The processor is
When input data is received, in a neural network operation process on the input data, an operation result for each channel is calculated with a synthetic scale parameter descaled based on a shift scaling factor corresponding to each channel. According to claim 1,
The artificial intelligence model is
a plurality of quantized weight values, a shift scaling factor for each channel included in each of the plurality of layers, and a scale parameter and a zero point parameter corresponding to each of the plurality of layers,
The scale parameter is
When each of the plurality of layers is quantized, a gradient between a value before quantization and a value after quantization is indicated,
The zero point parameter is
When each of the plurality of layers is quantized, an electronic device indicating a value after quantization of a zero value before quantization. 3. The method of claim 2,
The processor is
A value obtained based on a scale parameter of a current layer, a scale parameter of a layer immediately before the current layer, and a scale parameter of the plurality of weight values is inversely scaled based on a shift scaling factor corresponding to each channel, and the synthesized scale parameter to obtain, an electronic device. 4. The method of claim 3,
The processor is
and obtaining the synthesized scale parameter by shifting the obtained value based on a shift scaling factor corresponding to each channel. 4. The method of claim 3,
The obtained value is
The electronic device is inversely proportional to a scale parameter of the current layer and proportional to a scale parameter of a layer immediately before the current layer and a scale parameter of the plurality of weight values. 3. The method of claim 2,
The processor is
When it is identified that a plurality of channel-specific shift scaling factors included in each of the plurality of layers are included in the artificial intelligence model, the inverse scaling is performed. According to claim 1,
The electronic device is
An electronic device implemented as a Neural Processing Unit (NPU). According to claim 1,
The shift scaling factor corresponding to each channel is,
The electronic device is determined based on a weight value included in each channel and a weight value included in a layer including each channel. 9. The method of claim 8,
The shift scaling factor corresponding to each channel is,
The electronic device is determined based on a weight value having a largest magnitude in each channel and a weight value having a largest magnitude in a layer including each channel. A method for controlling an electronic device, comprising:
receiving input data; and
In the neural network calculation process for the input data using the artificial intelligence model, the calculation result for each channel of each of the plurality of layers constituting the artificial intelligence model is inversely scaled based on the shift scaling factor corresponding to each channel. and calculating with;
The artificial intelligence model is scaled based on a different shift scaling factor for each of a plurality of channels included in each of the plurality of layers and includes a plurality of weight values quantized for each of the plurality of layers. 11. The method of claim 10,
The artificial intelligence model is
a plurality of quantized weight values, a shift scaling factor for each channel included in each of the plurality of layers, and a scale parameter and a zero point parameter corresponding to each of the plurality of layers,
The scale parameter is
When each of the plurality of layers is quantized, a gradient between a value before quantization and a value after quantization is indicated,
The zero point parameter is
When quantizing each of the plurality of layers, a zero value before quantization indicates a value after quantization. 12. The method of claim 11,
The calculating step is
A value obtained based on a scale parameter of a current layer, a scale parameter of a layer immediately before the current layer, and a scale parameter of the plurality of weight values is inversely scaled based on a shift scaling factor corresponding to each channel, and the synthesized scale parameter to obtain, a control method. 13. The method of claim 12,
The calculating step is
and obtaining the synthesized scale parameter by shifting the obtained value based on a shift scaling factor corresponding to each channel. 13. The method of claim 12,
The obtained value is
The control method is inversely proportional to the scale parameter of the current layer and proportional to the scale parameter of the layer immediately before the current layer and the scale parameter of the plurality of weight values. 12. The method of claim 11,
The calculating step is
When it is identified that the shift scaling factors for each channel included in each of the plurality of layers are included in the artificial intelligence model, the operation is performed. 11. The method of claim 10,
The electronic device is
A control method implemented by a Neural Processing Unit (NPU). 11. The method of claim 10,
The shift scaling factor corresponding to each channel is,
The control method is determined based on a weight value included in each channel and a weight value included in a layer including each channel. 18. The method of claim 17,
The shift scaling factor corresponding to each channel is,
The control method is determined based on the largest weight value in each channel and the largest weight value in the layer including each channel.

Images