KR102596769B1 - Method for designing neural network and device for the same - Google Patents

Abstract

We disclose a method for determining the computational precision of each layer of a neural network operating on a computing device. This method includes calculating an index for each of the first set of layers included in the neural network, and assigning a first calculation precision to the upper N1 layers with smaller index values among the first set of layers. and allocating a second calculation precision having a lower precision than the first calculation precision to the upper N2 layers with large values of the indicator, and a layer to which the first calculation precision is assigned among the first set of layers. and storing information about the layer to which the second calculation precision is assigned in a memory.

Description

Neural network design method and device for the same {Method for designing neural network and device for the same}

The present invention relates to technology for designing an artificial intelligence neural network, and in particular, to technology for determining the precision of computational data used in each layer of a neural network.

A neural network is a processing module that converts input data into output data and is one of the functional modules that can be used in a computing device. A neural network may include a plurality of cascaded layers and other computational modules that are distinct from the layers, and information regarding this is disclosed in numerous documents.

Data input to each layer may be referred to as input activation, and data output from each layer may be referred to as output activation. The input activation and output activation are data composed of multiple values. The output activation output by one layer may be provided as input activation of another layer, or may be provided as input data of another calculation module distinct from the layer. For calculation in each layer, not only the input activation input to each layer, but also weights, which are data calculated according to the input activation and predetermined rules in each layer, may be required.

The operation of a neural network can be executed by loading command codes and data into the CPU. Alternatively, the operation of the neural network can be executed using a neural network accelerator, which is dedicated hardware designed and manufactured to support the operation of the neural network. In this case, the CPU and the neural network accelerator can cooperate with each other, and the CPU drives the neural network accelerator. executes the command code, and the neural network accelerator can read the necessary data from memory.

The raw data used for the calculation of the neural network may be, for example, data in the form of 32-bit floating point (FP32) or other forms of data. However, in order to reduce data memory traffic and perform lightweight operations, the data used in each layer of the neural network needs to be converted to fixed point (INT4, INT8, or INT16) form.

For example, the first input activation and first weight prepared for operation in the first layer may need to be prepared in fixed-point form with the first number of bits. If the first input activation and first weight prepared for operation in the first layer are not in fixed-point form with the first number of bits, it is necessary to convert them into fixed-point form with the first number of bits. . In this way, the technique of approximating data in the form of floating points to a fixed point form using a small number of bits for the purpose of reducing data memory traffic and making lightweight calculations can be referred to as neural network quantization.

When a given neural network includes multiple layers, all of the multiple layers can be quantized with the same precision (single precision neural network quantization). At this time, a first case in which all of the plurality of layers are quantized into a fixed point form with a first number of bits can be compared with a second case in which all of the plurality of layers are quantized in a fixed point form with a second number of bits. Assuming that the second number of bits is greater than the first number of bits, the first case is more advantageous in terms of lightening the neural network, but is more disadvantageous in terms of quantization error, and the second case is more disadvantageous in terms of lightening the neural network, but in terms of quantization error, It is more advantageous.

Unlike the above, some of the plurality of layers of a given neural network may be quantized in fixed-point form with a first number of bits, and others may be quantized in fixed-point form with a second number of bits (mixed-precision neural network quantization). By doing this, a more advantageous effect can be achieved overall when considering both quantization error and neural network lightweight perspectives. At this time, the first group of layers to be quantized in fixed point form (first precision) of the first number of bits and the second group of layers to be quantized in fixed point form (second precision) of the second number of bits are selected and determined. Must be able to.

The present invention seeks to provide a technique for determining first group layers to be quantized with first precision and second group layers to be quantized with second precision among a plurality of layers included in a given neural network.

According to one aspect of the present invention, a method for determining the computational precision of each layer of a neural network operating on a computing device may be provided. The method includes calculating, by a computing device, an index (SQNR _Avg ) for each of a first set of layers included in a neural network; The computing device allocates a first operation precision to the upper N1 layers with smaller index values among the layers of the first set and performs the first operation on the upper N2 layers with larger index values. Allocating a second arithmetic precision that is lower than the precision; and storing, by the computing device, information about a layer to which the first computational precision is assigned and information about a layer to which the second computational precision is assigned among the first set of layers in a memory. Here, information about the layer to which the first computational precision is assigned may be information identifying the layer. Additionally, information about the layer to which the second calculation precision is assigned may also be information identifying the layer.

At this time, the step of calculating the indicator for a specific layer included in the neural network includes the output of the specific layer when the first data calculated in the specific layer is quantized in a fixed point form with a first number of bits. When the first output activation and the first data are quantized in fixed-point form with a second number of bits, a value generated based on the difference value or distance between the second output activation output by the specific layer is used as the specific output. A correction step may be included using the size of the output activation of the layer.

At this time, the indicator for the specific layer included in the neural network is the first output activation output by the specific layer when the first data calculated in the specific layer is quantized in fixed point form with the first number of bits. When the first data is quantized in a fixed-point format with a second number of bits, the smaller the difference between the second output activations output by the specific layer, the larger the value.

At this time, the indicator for the specific layer included in the neural network is the first output activation output by the specific layer when the first data calculated in the specific layer is quantized in fixed point form with the first number of bits. When the first data is quantized in fixed-point form with a second number of bits, the smaller the distance between the second output activation output by the specific layer, the larger the value.

At this time, the indicator for a specific layer included in the neural network may have a smaller value as the size of output activation of the specific output layer increases.

At this time, calculating the indicator includes: acquiring, by the computing device, the size (T) of the output activation of the kth layer included in the neural network; The computing device prepares first data used in the k-th layer in a fixed-point format with a first number of bits, and outputs a first output from the k-th layer using the first data prepared with the first number of bits. Generate activation (v _i ) (i=1, 2, 3, ..., T), and a first sum (i), which is the sum of the squares of each element of the first output activation ( ) The first step of calculating; The computing device prepares the first data in a fixed-point format with a second number of bits, and performs a second output activation (qv _i ) from the kth layer using the first data prepared with the second number of bits. Generating a second sum (which is the sum of the squares of the difference between each element of the first output activation and each element of the corresponding second output activation) ) calculating; The computing device provides a first log value, which is a log value of the ratio of the first sum to the second sum ( ), and calculating a second log value (log ₁₀ T), which is a log value for the size (T) of the output activation of the kth layer; and determining, by the computing device, a quantization error index (SQNR[k] _Avg ) for the kth layer by subtracting a value proportional to the second log value from a value proportional to the first log value. May include ;. Here, i (i=1, 2, 3, ..., T) is an identification index that distinguishes each of the T activations.

Alternatively, the indicator (SQNR _Avg ) regarding a specific layer of the neural network may satisfy Equation 1.

[Formula 1]

Here, T is the size of the output activation of a specific layer.

Here, v _i is the value of the i-th data with index i among the output activations of the specific layer, when the first data operated on the specific layer is quantized in fixed-point form with the first number of bits.

Here, qv _i is the index i among the output activations of the specific layer when the first data operated in the specific layer is quantized in fixed-point form with a second number of bits smaller than the first number of bits. It is the value of the ith data.

Here, α and β are constants.

And here, am.

At this time, the first data may include the kth input activation input to the kth layer and the kth weight used in the kth layer.

At this time, the data processed in the upper predetermined number of layers where the indicator value is small is calculated in the state expressed in fixed-point form of the first bit, and the data processed in the upper predetermined number of layers where the indicator value is large is calculated. The data to be processed is operated in a fixed-point format of the second bits, and the number of the first bits may be greater than the number of the second bits.

According to another aspect of the present invention, a neural network calculation method for calculating output activation of a layer included in a neural network operating on a computing device may be provided. This neural network calculation method includes: acquiring, by the computing device, computational precision assigned to the k-th layer of the neural network, a k-th input activation to be input to the k-th layer, and a k-th weight corresponding to the k-th layer; determining, by the computing device, whether the kth input activation and the kth weight are expressed in the number of bits corresponding to the computational precision assigned to the obtained kth layer; If the k-th weight is not expressed in the number of bits corresponding to the computational precision assigned to the k-th layer, the computing device sets the k-th input activation and the k-th weight to correspond to the obtained computational precision. Converting the number of bits to have a fixed-point format; and calculating, by the computing device, the kth output activation of the kth layer using the converted kth input activation and the kth weight.

At this time, the computational precision allocated to the kth layer of the neural network includes calculating an index (SQNR _Avg ) for each of the first set of layers included in the neural network; Among the layers of the first set, a first calculation precision is assigned to the top N1 layers with small indicator values, and a precision lower than the first calculation precision is assigned to the top N2 layers with large indicator values. Allocating a second computational precision; and storing, in a memory, information about the layer to which the first arithmetic precision is assigned and information about the layer to which the second arithmetic precision is assigned among the first set of layers. This may have been decided.

At this time, the indicator (SQNR _Avg ) may satisfy Equation 1 above.

According to another aspect of the present invention, a neural network calculation method for calculating the output activation of a layer included in the neural network may be provided. The neural network calculation method includes calculating, by a first computing device, an index (SQNR _Avg ) for each of the first set of layers included in the neural network; The first computing device allocates a first calculation precision to the upper N1 layers with smaller index values among the layers of the first set, and the first calculation precision to the upper N2 layers with larger index values. Allocating a second arithmetic precision that is lower than the first arithmetic precision; storing, by the first computing device, information about a layer to which the first computational precision is assigned and information about a layer to which the second computational precision is assigned among the first set of layers in a memory; Obtaining, by a second computing device, computational precision assigned to the k-th layer of the neural network, a k-th input activation to be input to the k-th layer, and a k-th weight corresponding to the k-th layer; determining, by the second computing device, whether the kth input activation and the kth weight are expressed in the number of bits corresponding to the computational precision assigned to the obtained kth layer; If the second computing device determines that the k-th weight is not expressed in a number of bits corresponding to the computational precision assigned to the k-th layer, the k-th input activation and the k-th weight are converted to the obtained computational precision. Converting the number of bits corresponding to to have a fixed-point format; and calculating, by the second computing device, the kth output activation of the kth layer using the converted kth input activation and the kth weight. Includes.

According to another aspect of the present invention, a computing device capable of accessing a non-volatile recording medium and including a processing unit may be provided. In the non-volatile recording medium, information about the neural network, a first command code for calculating an index for each layer, and a second command code for determining the calculation precision for each layer are recorded. The processing unit reads and executes the first command code, thereby calculating an index (SQNR _Avg ) for each of the first set of layers included in the neural network; By reading and executing the second command code, the first calculation precision is assigned to the upper N1 layers with smaller index values among the layers of the first set, and to the upper N2 layers with larger index values. assigning a second arithmetic precision that is lower than the first arithmetic precision; and storing information about the layer to which the first computational precision is assigned and information about the layer to which the second computational precision is assigned among the first set of layers in the non-volatile recording medium. .

According to another aspect of the present invention, a computing device capable of accessing a non-volatile recording medium and including a processing unit may be provided. In the non-volatile recording medium, information about the neural network and computational precision assigned to the kth layer of the neural network are recorded. The processing unit acquires computational precision assigned to the k-th layer of the neural network, a k-th input activation to be input to the k-th layer, and a k-th weight corresponding to the k-th layer; determining whether the kth input activation and the kth weight are expressed in the number of bits corresponding to the computational precision assigned to the obtained kth layer; If the k-th weight is not expressed in the number of bits corresponding to the computational precision assigned to the k-th layer, the k-th input activation and the k-th weight are fixed to the number of bits corresponding to the obtained computational precision. Converting to have decimal form; And calculating the kth output activation of the kth layer using the converted kth input activation and the kth weight.

According to the present invention, it is possible to provide a technique for determining first group layers to be quantized with first precision and second group layers to be quantized with second precision among a plurality of layers included in a given neural network.

Figure 1 conceptualizes only the layers included in the neural network among the given neural network structure.
Figure 2 shows the configuration of a computing device according to an embodiment of the present invention used to design a neural network used by the computing device.
Figure 3 is a flowchart showing a process for determining computational precision for each layer of a neural network provided according to an embodiment of the present invention.
Figure 4 is a flowchart showing a process for calculating indices for each layer of a neural network according to an embodiment of the present invention.
Figure 5 shows the main structure of some of the computing devices used in one embodiment of the present invention.
Figure 6 is a flowchart showing a method of calculating output activation provided according to an embodiment of the present invention.
Figure 7 shows the configuration of a computing device that executes a neural network calculation method according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described with reference to the attached drawings. However, the present invention is not limited to the embodiments described herein and may be implemented in various other forms. The terms used in this specification are intended to aid understanding of the embodiments and are not intended to limit the scope of the present invention. Additionally, as used herein, singular forms include plural forms unless phrases clearly indicate the contrary.

Figure 1 conceptualizes only the layers included in the neural network among the given neural network structure.

The neural network (1000) may include a plurality of layers (10[k]) (k=1, 2, 3, ..., K). The kth input activation is input to the kth layer (10[k]). And, the kth output activation can be output from the kth layer (10[k]). In order to generate the kth output activation, the kth input activation and the kth layer (10[k]) are prepared in advance for the kth layer (10[k]). An operation using the kth weight may be executed. At this time, the values constituting the kth input activation, the kth output activation, and the kth weight may be quantized in a fixed point form with the kth number of bits. .

In one embodiment, the kth number of bits may be the same for all k values (k=1, 2, 3, ..., K).

In another embodiment, the k-th number of bits for the k values in the first group may all have the first number of bits, and the k-th number of bits for the k values in the second group may all have the second number of bits. . At this time, the first number of bits may be greater than the second number of bits.

Figure 2 shows the configuration of a computing device used to design a neural network used by the computing device.

Computing device 11 may include a non-volatile recording medium 12, a device interface 13, and a processing unit 14. The non-volatile recording medium 12 may be soldered to the computing device 11 or may be a removable memory. The processing unit 14 may be a CPU or the like. The device interface may include a data bus, command bus, and bus controller.

The non-volatile recording medium 12 may store neural network information including information about the structure of the neural network. The neural network information may include information necessary for processing input data input to the neural network, such as the number of layers of the neural network, the input/output conversion function of each layer, and the functions of calculation modules other than the layer.

In one embodiment of the present invention, an index that arithmetically evaluates the characteristics of each layer of the neural network is defined. The specific definition of this indicator is explained in detail in Equation 2 below.

The non-volatile recording medium 12 may include a first command code (index calculation command code for each layer), which is a command code for calculating the index for each layer of the neural network.

The non-volatile recording medium 12 may include a second command code (computation precision determination command code for each layer) that determines the calculation precision allocated to each layer of the neural network using the calculated indices.

For example, in the kth layer, the kth output activation is generated through an operation that multiplies or adds the kth input activation and the kth weight. At this time, the kth input activation and the kth weight are calculated by expressing them in fixed point form with the kth number of bits. At this time, it is necessary to determine in advance what the k-th number of bits is. The larger the value, the higher the calculation precision and the lower the quantization error. Therefore, determining the computational precision can be understood as determining a specific value of the kth number of bits.

The neural network information may include information indicating computational precision for each layer, and this information may be generated or updated by executing the first command code and the second command code.

The neural network information, the first command code, and the second command code stored in the non-volatile recording medium 12 may be loaded into the processing unit 14 through the device interface 13.

The processing unit 14 may execute an index calculation process for each layer by executing the first command code, and execute a process for determining calculation precision for each layer by executing the second command code.

Figure 3 is a flowchart showing a process for determining computational precision for each layer of a neural network provided according to an embodiment of the present invention.

In step S110, the computing device 11 may calculate an indicator (SQNR[k] _Avg ) regarding quantization error for each of the selected first set of layers of a given neural network.

Here, the first set of layers may be all layers included in the neural network, or may be some of all layers. In one embodiment, the criteria for selecting the first set of layers may not be particularly limited.

The definition of the quantization error indicator (SQNR[k] _Avg ) is explained through Equation 2 below.

If the number of layers in the first set is N, N indicators for the quantization error (SQNR[k] _Avg ) can also be generated.

In step S120, the computing device 11 allocates a first quantization precision (high precision, high number of bits) to the upper N1 layers with small index values among the first set of layers, and The second quantization precision (low precision, low number of bits) can be assigned to the upper N2 layers with large index values.

At this time, when the number of layers in the first set is N, N1+N2=N or N1+N2<N.

For example, the data processed in the upper N1 layers with small index values can be calculated in an 8-bit fixed-point format, and the data processed in the upper N2 layers with large index values can be calculated in 4 It can be operated on with bits expressed in fixed-point form.

In step S130, the computing device 11 may include information about the layer to which the first quantization precision is assigned and information about the layer to which the second quantization precision is assigned to the neural network information. This neural network information can be stored in the non-volatile recording medium 12.

In this specification, quantization precision (kth quantization precision) may also be referred to as arithmetic precision (kth arithmetic precision).

In one embodiment, as an indicator of the above-described quantization error, SQNR _Avg , a metric for a specific layer, can be defined as Equation 2.

[Formula 2]

In [Formula 2], T is the size of the output activation of the specific layer, that is, the number of data constituting the output activation,

v _i is the i th value with index i among the output activations of the specific layer, when all data operated on in the specific layer are quantized to the first precision or quantized in fixed point form with the first number of bits. is the value of the data,

qv _i is the i th value with index i among the output activations of the specific layer, when all data operated in the specific layer is quantized with second precision or quantized in fixed point form with the second number of bits. is the value of the data,

v _{i -} qv _i can be referred to as the difference value of quantization error or quantization error difference,

α, β are constants.

Here, the first precision may be greater than the second precision, and the first number of bits may be greater than the second number of bits.

In other words, Equation 2 compares the results of executing the operation of the specific layer twice. In the first operation, the first input activation and first weight quantized with the first precision are provided to the specific layer to calculate the first output activation (v i ), and in the second operation, the first output activation (v _i ) is calculated to the specific layer. The second output activation (v _i ) is calculated by providing the second input activation and second weight quantized with . At this time, the first input activation and the second input activation are the same input activation, but are quantized with different precision. And the first weight and the second weight are the same weight, but are quantized with different precision.

For example, {the first number of bits, the second number of bits} may be {16, 8}, {16, 4}, or {8, 4}.

From Equation 2, Equations 3 and 4 below are established.

[Formula 3]

[Formula 4]

When the specific layer is the k-th layer (10[k]), SQNR _Avg in Equation 2 can be expressed as SQNR[k] _Avg .

The fact that the SQNR in Equation 3 is large, that is, v _i -qv _i according to the adjustment of the number of bits, is small, means that the quantization error of output activation is small even if the number of bits representing the data used in the operation in the corresponding layer is changed to a smaller value. This means that it does not increase much. Therefore, for a layer with a large SQNR, this may mean that it is acceptable to allow the data used in calculations in that layer to be quantized (small precision) into a smaller number of bits.

On the other hand, for a layer with a small SQNR, this may mean that the data used for calculations in that layer must be quantized (high precision) with a larger number of bits.

In other words, it is desirable that data used in calculations in layers with large SQNR values be quantized with a small number of bits, and data used in calculations in layers with small SQNR values should be quantized with a large number of bits. do.

At this time, in a preferred embodiment of the present invention, a correction value SQNR _Avg obtained by reducing the calculated value of SQNR according to a predetermined rule can be used (Equation 2). The correction term for the reduction is -β·log ₁₀ T given in Equation 4. According to this, the SQNR value of a layer is corrected to be reduced more as the output activation size (T) of the layer is larger, and is corrected to be reduced less as the output activation size (T) is smaller.

In the case of a layer where the size of the output activation is relatively large, even if the SQNR has a large value, the value is greatly reduced by the correction term. In other words, if the size of the output activation is a relatively large layer, it is processed to increase the possibility of quantization with a large number of bits (high precision).

On the other hand, in the case of a layer where the size of the output activation is relatively small, even if the SQNR has a small value, the degree to which the value is reduced by the correction term is small. In other words, if the size of the output activation is a relatively small layer, the layer is processed to increase the possibility of quantization with a low number of bits (low precision).

In other words, the above-mentioned correction term is a correction term that increases the possibility of quantization processing with a high number of bits (high precision) when the size of the output activation is large rather than when the size of the output activation is relatively small.

In one embodiment, step S140 may be executed after step S130. In step S140, weight values corresponding to each layer of the neural network may be stored in the non-volatile memory. At this time, the stored weight values may be included in the neural network information. At this time, if the first quantization precision is assigned to the k-th layer, the weight values corresponding to the k-th layer may be quantized and stored in fixed-point format with the first number of bits. In contrast, if the second quantization precision is assigned to the k-th layer, the weight values corresponding to the k-th layer may be quantized and stored in fixed-point format with the second number of bits.

In another embodiment, weight values corresponding to each layer of the neural network may not be included in the neural network information stored in the non-volatile recording medium 12. That is, the quantization precision assigned to each layer of the neural network is included in the neural network information, but weight values corresponding to each layer may be prepared through a separate weight determination process. In the weight determination process, the number of bits representing weight values corresponding to each layer can be determined using the quantization precision for each layer included in the neural network information.

The indicator for a specific layer provided according to an embodiment of the present invention presented in Equation 2 is, when the first data operated on the specific layer is quantized in a fixed-point format with the first number of bits, the specific layer The difference value (v _i -qv _i ) or distance between the first output activation output and the second output activation output by the specific layer when the first data is quantized in fixed-point form with a second number of bits. ( ) can be considered as having been corrected using the size (T) of the output activation of the specific output layer.

The indicator for a specific layer provided according to an embodiment of the present invention presented in Equation 2 is, when the first data operated on the specific layer is quantized in a fixed-point format with the first number of bits, the specific layer The difference value (v _i -qv _i ) or distance between the first output activation output and the second output activation output by the specific layer when the first data is quantized in fixed-point form with a second number of bits. ( ), the smaller it can be, the larger the value can be.

Additionally, the indicator for a specific layer provided according to an embodiment of the present invention shown in Equation 2 may have a smaller value as the size of output activation of the specific output layer increases.

Figure 4 is a flowchart showing a process for calculating indices for each layer of a neural network according to an embodiment of the present invention.

The flow chart shown in FIG. 4 embodies the step (S110) of FIG. 3.

In step S111, the computing device 11 may obtain the size of the output activation of the kth layer included in the neural network from the neural network information.

For example, the size of the output activation may be T shown in Equation 2.

Then, in step S112, computing device 11:

① Prepare the first data used to generate the kth output activation in the kth layer in a fixed point form with the first number of bits,

② Generate a first output activation from the kth layer using the first data having a fixed point form with the first number of bits, and

③ The first sum, which is the sum of the squares of each element of the first output activation, can be calculated.

For example, the first output activation may be v _i (i=1, 2, 3, ..., T) shown in Equation 2, and the first sum is shown in Equation 2. It can be.

Here, the first data may include a kth input activation input to the kth layer and a kth weight used in the kth layer.

In step S113, the computing device 11:

① The first data used to generate the kth output activation in the kth layer is prepared in fixed-point form with a second number of bits different from the first number of bits,

② Generate a second output activation from the kth layer using the first data in a fixed-point format with the second number of bits, and

③ The second sum, which is the sum of the squares of the differences between each element of the first output activation and each element of the corresponding second output activation, can be calculated.

The k-th layer and first data in step S113 are the same as the k-th layer and first data in step S112. However, the number of bits representing the first data in step S113 is different from the number of bits representing the first data in step S112.

For example, the second output activation is qv _i (i=1, 2, 3, ..., T) shown in Equation 2, and each element of the first output activation and the corresponding second output activation are The difference between the elements is v _i -qv _i presented in Equation 2, and the second sum is presented in Equation 2. It can be.

Then, in step S114, computing device 11:

① Calculate the first log value, which is the logarithm value of the ratio of the first sum to the second sum, and

② A second log value, which is a log value for the size (T) of the output activation, can be calculated.

For example, the ratio of the first sum to the second sum is shown in Equation 2 And the first log value is shown in Equation 2 , and the second log value may be log ₁₀ T shown in Equation 2.

Next, in step S115, the computing device 11 subtracts a value proportional to the second log value from a value proportional to the first log value and provides an index regarding the quantization error for the kth layer ( SQNR[k] _Avg ) can be determined.

For example, the value proportional to the first log value may be SQNR shown in Equation 3, and the value proportional to the second log value may be βㅇlog ₁₀ T shown in Equation 4.

Figure 5 shows the main structure of some of the computing devices used in one embodiment of the present invention.

Computing device 1 may be the same as computing device 11 shown in FIG. 2 or may be different. The computing device 11 of FIG. 2 may be a device used by a designer who designs the structure and operation rules of a neural network, and the computing device 1 of FIG. 5 may be a device that performs a predetermined operation using the designed neural network. It could be a device.

The computing device 1 includes a dynamic random access memory (DRAM) 130, a hardware accelerator 110, a bus 700 connecting the DRAM 130 and the hardware accelerator 110, and other devices connected to the bus 700. It may include hardware 99 and a main processor 160. Here, DRAM 130 may be referred to as memory 130.

In addition, the computing device 1 may further include a power supply unit, a communication unit, a user interface, a storage unit 170, and peripheral device units not shown. The bus 700 may be shared by the hardware accelerator 110, other hardware 99, and the main processor 160.

The hardware accelerator 110 includes a DMA unit (Direct Memory Access part) 20, a control unit 40, an internal memory 30, an input buffer 650, a data operation unit 610, and an output buffer 640. can do.

Some or all of the data temporarily stored in the internal memory 30 may be provided from the DRAM 130 through the bus 700. At this time, in order to move data stored in the DRAM 130 to the internal memory 30, the control unit 40 and the DMA unit 20 may control the internal memory 30 and the DRAM 130.

Data stored in the internal memory 30 may be provided to the data calculation unit 610 through the input buffer 650.

Output values generated by the operation of the data calculation unit 610 may be stored in the internal memory 30 through the output buffer 640. The output values stored in the internal memory 30 may be written to the DRAM 130 under the control of the control unit 40 and the DMA unit 20.

The control unit 40 can collectively control the operations of the DMA unit 20, the internal memory 30, and the data operation unit 610.

In one implementation, the data calculation unit 610 may perform a first calculation function during a first time period and a second calculation function during a second time period. For example, during the first time period, an arithmetic function is performed to calculate the output activation of the first layer from the input activation of the first layer of the neural network, and during the second time period, the calculation function of the second layer is calculated from the input activation of the second layer of the neural network. The calculation function that calculates output activation can be performed.

In FIG. 5, one data operation unit 610 is shown within the hardware accelerator 110. However, in a modified embodiment not shown, a plurality of data calculation units 610 shown in FIG. 5 may be provided in the hardware accelerator 110 to perform operations requested by the control unit 40 in parallel. there is.

In one implementation, the data calculation unit 610 may output the output data sequentially according to a given order over time rather than all at once.

The memory 130 of FIG. 5 may store the neural network information completed in step S130 of FIG. 3 and stored in the non-volatile recording medium 12 of FIG. 2.

The neural network information may include information about the layer to which the first quantization precision is assigned and information about the layer to which the second quantization precision is assigned. Accordingly, information regarding whether the quantization precision assigned to the kth layer of the neural network is the first quantization precision or the second quantization precision can be prepared.

In one time period, the hardware accelerator 110 may calculate the kth output activation that the kth layer should output among the neural network information stored in the memory 130. For this purpose, the kth input activation and kth weight input to the kth layer may be stored in the memory 130. In the one time period, the hardware accelerator 110 reads the kth input activation and kth weight stored in the memory 130 through the bus 700 and temporarily stores them in the internal memory 30. , the kth output activation can be calculated using the data calculation unit 610. This series of processes may be controlled by the control unit 40, and at least some of the command codes that operate the control unit 40 may be provided by the main processor 160.

In one embodiment, the kth weight and/or the kth input activation stored in the internal memory 30 or the memory 130 are used to calculate the kth output activation of the kth layer. It can have a fixed-point number of bits. At this time, if it is confirmed that the quantization precision allocated to the k-th layer corresponds to the second number of bits, the control unit 40 controls the k-th weight and/or the k-th weight having a fixed point form of the first number of bits. Input activation may be converted to a fixed point form of the second number of bits. Next, the data calculation unit 610 may calculate the kth output activation using the kth weight and/or the kth input activation having a fixed point form of the converted second number of bits.

Figure 6 is a flowchart showing a method of calculating output activation provided according to an embodiment of the present invention.

In one embodiment, each step shown in Figure 6 may be executed in hardware accelerator 110 of Figure 5.

In step S210, the hardware accelerator 110 may read some or all of the neural network information from the memory 130. The neural network information may include the structure of the neural network.

In step S220, the hardware accelerator 110 may check the quantization precision assigned to the kth layer from the neural network information.

In step S230, the hardware accelerator 110 may read the kth input activation to be input to the kth layer of the neural network and the kth weight corresponding to the kth layer from memory. Step S230 may have already been executed in step S210.

In step S240, the hardware accelerator 110 may check whether the kth input activation and the kth weight are expressed in the number of bits corresponding to the confirmed quantization precision. If the kth weight is expressed in the number of bits corresponding to the confirmed quantization precision, step S260 can be performed. Otherwise, step S250 can be performed.

In step S250, the hardware accelerator 110 may convert the kth input activation and the kth weight to have a fixed point form with the number of bits corresponding to the confirmed quantization precision.

In step S260, the hardware accelerator 110 may calculate the kth output activation of the kth layer using the kth input activation and the kth weight.

The calculated k-th output activation may be used while temporarily stored in the internal memory 30, or may be stored in the memory 130, which is a non-volatile recording device.

Figure 7 shows the configuration of a computing device that executes a neural network calculation method provided according to an embodiment of the present invention.

Computing device 21 may include a non-volatile recording medium 22, a device interface 23, and a processing unit 24. The non-volatile recording medium 22 may be soldered to the computing device 21 or may be a removable memory. The processing unit 24 may be a CPU or the like. The device interface may include a data bus, command bus, and bus controller.

The non-volatile recording medium 22 contains information about the neural network, computational precision assigned to the k-th layer of the neural network, k-th input activation to be input to the k-th layer, and k-th weight corresponding to the k-th layer. It may be stored. This information may be generated using the computing device 11 shown in FIG. 2.

In the non-volatile recording medium 22, the processing unit 24 stores the computational precision assigned to the k-th layer of the neural network, the k-th input activation to be input to the k-th layer, and the k-th layer corresponding to the k-th layer. Obtaining k weights; determining whether the kth input activation and the kth weight are expressed in the number of bits corresponding to the computational precision assigned to the obtained kth layer; If the k-th weight is not expressed in the number of bits corresponding to the computational precision assigned to the k-th layer, the k-th input activation and the k-th weight are fixed to the number of bits corresponding to the obtained computational precision. Converting to have decimal form; and calculating the kth output activation of the kth layer using the converted kth input activation and the kth weight. A process for executing a neural network calculation process (i.e., neural network calculation method) including. 3A program in which command codes (neural network operation command codes) are recorded may be stored.

The processing unit 24 may be configured to execute the neural network calculation method by reading and executing the third command code through the device interface 23.

The computing device 11 of FIG. 2 and the computing device 21 of FIG. 7 may be the same device or may be separate devices.

When the computing device 11 of FIG. 2 and the computing device 21 of FIG. 7 are separate devices, the computing device 11 of FIG. 2 and the computing device 21 of FIG. 7 cooperate with each other to form one computing system. can do. At this time, the computing system can execute all methods of the present invention described above. At this time, the computing device 11 of FIG. 2 and the computing device 21 of FIG. 7 may exchange information through a local network or may exchange information through a remote network such as a metropolitan network.

By using the above-described embodiments of the present invention, those in the technical field of the present invention will be able to easily make various changes and modifications without departing from the essential characteristics of the present invention. The contents of each claim in the patent claims can be combined with other claims without reference within the scope that can be understood through this specification.

Claims (18)

A method of determining the computational precision of each layer of a neural network operating on a computing device,
Calculating, by the computing device, an index for each of a first set of layers included in the neural network;
The computing device allocates a first operation precision to the upper N1 layers with smaller index values among the layers of the first set and performs the first operation on the upper N2 layers with larger index values. Allocating a second calculation precision having a precision different from the precision; and
storing, by the computing device, information about a layer to which the first computational precision is assigned and information about a layer to which the second computational precision is assigned among the first set of layers in a memory;
Includes,
The maximum value of the indicators of the N1 layers is smaller than the minimum value of the indicators of the N2 layers,
The step of calculating the indicator for a specific layer among the layers of the first set is,
The computing device calculates a first set of values of the first output activation output by the specific layer when the first data operated on the specific layer is quantized in fixed-point form with a first number of bits. step,
The computing device calculates a second set of values of the second output activation output by the specific layer when the first data operated on the specific layer is quantized in fixed-point form with a second number of bits. step, and
calculating, by the computing device, the indicator for the specific layer using a difference between the first set of values and the second set of values;
Including,
How to determine calculation precision. The method of claim 1, wherein in calculating the indicator for the specific layer using a difference between the values of the first set and the values of the second set, the indicator for the specific layer is the value of the first set. A method for determining calculation precision, characterized in that it is calculated using the distance between the value and the second set of values. The method of claim 2, wherein the smaller the distance between the first set of values and the second set of values, the larger the value of the indicator calculated for the specific layer. method. The method of claim 2, wherein the index for the specific layer is calculated further using the size of the number of data constituting the output activation of the specific layer. The method of claim 1, wherein the smaller the size of the number of data constituting the output activation of the specific layer, the larger the value of the indicator calculated for the specific layer. method. According to paragraph 1,
The step of calculating the above indicator is,
Obtaining, by the computing device, a size of output activation of a kth layer included in the neural network;
The computing device prepares first data used in the k-th layer in a fixed-point format with a first number of bits, and outputs a first output from the k-th layer using the first data prepared with the first number of bits. A first step of generating an activation and calculating a first sum, which is the sum of the squares of each element of the first output activation;
The computing device prepares the first data in a fixed-point format with a second number of bits and generates a second output activation from the kth layer using the first data prepared with the second number of bits, and calculating a second sum, which is the sum of the squares of differences between each element of the first output activation and each element of the corresponding second output activation;
The computing device calculates a first log value, which is a log value of the ratio of the first sum to the second sum, and a second log value, which is a log value of the size of the output activation of the kth layer. steps; and
determining, by the computing device, an index regarding quantization error for the kth layer by subtracting a value proportional to the second log value from a value proportional to the first log value;
Including,
How to determine calculation precision. The method of claim 1, wherein the indicator (SQNR _Avg ) regarding a specific layer of the neural network satisfies Equation 1,
[Formula 1]

However, T is a size related to the number of data constituting the output activation of a specific layer;
v _i is the value of the ith data with index i among the output activations of the specific layer, when the first data operated on the specific layer is quantized in fixed-point form with the first number of bits;
qv _i is i with index i among the output activations of the specific layer when the first data operated in the specific layer is quantized in fixed-point form with a second number of bits smaller than the first number of bits. value of the second data;
α, β are constants; and
,
How to determine calculation precision. The method of claim 6 or 7, wherein the first data includes input activation input to the specific layer and a weight used in the specific layer. The method of claim 1, wherein the first number, which is the number of bits representing data processed in the upper N1 layers with small index values in fixed-point form, is processed in the upper N2 layers with large index values. A method for determining calculation precision, characterized in that the second number is greater than the number of bits representing the data in fixed-point form. The method of claim 1, wherein the second calculation precision is lower than the first calculation precision. delete delete delete delete delete delete A computing device capable of accessing a non-volatile recording medium and including a processing unit, comprising:
In the non-volatile recording medium, information about the neural network, a first command code for calculating an index for each layer, and a second command code for determining the calculation precision for each layer are recorded,
The processing unit,
calculating an index for each of the first set of layers included in the neural network by reading and executing the first command code;
By reading and executing the second command code, the first calculation precision is assigned to the upper N1 layers with smaller index values among the layers of the first set, and to the upper N2 layers with larger index values. assigning a second arithmetic precision having a different precision from the first arithmetic precision; and
storing information about a layer to which the first computational precision is assigned and information about a layer to which the second computational precision is assigned among the first set of layers in the non-volatile recording medium;
is set to run,
The maximum value of the indicators of the N1 layers is smaller than the minimum value of the indicators of the N2 layers,
The step of calculating the indicator for a specific layer among the layers of the first set is,
Calculating a first set of values of the first output activation output by the specific layer when the first data operated on the specific layer is quantized in fixed-point form with a first number of bits,
Calculating a second set of values of the second output activation output by the specific layer when the first data operated on the specific layer is quantized in fixed-point form with a second number of bits, and
Calculating the indicator for the specific layer using the difference between the first set of values and the second set of values,
Including,
Computing device. The computing device of claim 17, wherein the second calculation precision is lower than the first calculation precision.

Images