KR20190076916A - Method and apparatus for outlier-aware accelerating neural networks - Google Patents

Abstract

Presented are a method for accelerating data processing in a neural network and an apparatus thereof. The apparatus may comprise: a control unit performing quantization on data based on the characteristics of the data on a neural network and separating and performing calculation on the data according to expression form of the quantized data; and a memory storing the quantized data.

Description

TECHNICAL FIELD [0001] The present invention relates to a method and apparatus for accelerating neural networks,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for accelerating a data rate by classifying data according to a format of data on a neural network .

Deep neural networks are generally divided into convolutional neural networks (CNNs) using convolution and recurrent neural networks (RNN) using matrix-vector multiplication. CNN is known to be suitable for image processing, and RNN is suitable for continuous data processing. In each layer of CNN and RNN, output is obtained by convolution or multiplication of input data and weight, and then an active function (generally, ReLU or sigmoid) is applied to it to obtain the output of the layer.

These deep neural networks require many computations, most of which are convolution and matrix-vector or matrix-matrix calculations.

In order to perform convolution and matrix multiplication calculations quickly, we quantize the data to reduce the amount of memory access and computation, but the accuracy of existing data can not be maintained.

In the related art, Korean Patent Laid-Open No. 10-2001-0105425 discloses a method for predicting the operation of an MFSFET device which is a nonvolatile memory device and an adaptive learning circuit of a neuro-device using the device, The operation of the nonvolatile memory device in which the current flow is controlled by switching the polarization of the nonvolatile memory device can not be solved.

Therefore, a technique for solving the above-described problems is required.

On the other hand, the background art described above is technical information acquired by the inventor for the derivation of the present invention or obtained in the derivation process of the present invention, and can not necessarily be a known technology disclosed to the general public before the application of the present invention .

The embodiments disclosed herein provide a neural network acceleration method and apparatus for applying quantization differently based on the distribution of data so that the data format can be configured with a small number of bits while maintaining the quality of the output of the neural network .

The embodiments disclosed herein provide a method and apparatus for accelerating a neural network in which a certain amount of outlier data having a large value among data of a neural network is calculated with high accuracy and the remaining data is calculated with a low accuracy to reduce the amount of calculation .

The embodiments disclosed herein are aimed at providing a neural network acceleration method and apparatus that reduces the amount of memory access data by varying the quantization for the data of the neural network.

According to an embodiment of the present invention, there is provided an apparatus for accelerating data processing on a neural network, comprising: a quantizer for quantizing the data based on characteristics of data on the neural network; A controller for separating and performing calculations on the data according to a representation format of the data, and a memory for storing the quantized data.

According to another embodiment, a method for accelerating data processing on a neural network, the method comprising: performing quantization on the data based on characteristics of the data on the neural network; storing the quantized data And separating and performing the calculation on the data according to a representation format of the quantized data.

According to another embodiment, there is provided a computer readable medium having recorded thereon a program for performing a neural network acceleration method, the method comprising: performing quantization on the data based on characteristics of data on the neural network; Storing the quantized data, and separating and performing calculations on the data according to a representation format of the quantized data.

According to another embodiment, a computer program stored on a recording medium for performing a neural network acceleration method, the neural network acceleration method comprising the steps of: Performing quantization, storing the quantized data, and separating and performing calculation on the data according to a representation format of the quantized data.

According to any one of the above-mentioned problem solving means, a neural network acceleration method and apparatus are proposed in which quantization is applied differently based on the distribution of data so that the data format can be configured with a small number of bits while maintaining the output quality of the neural network can do.

Further, according to any one of the above-mentioned means for solving the above-mentioned problems, a neural network acceleration method and apparatus for calculating a certain amount of outlier data having a large value among data of a neural network with high precision and calculating the remaining data with low precision, .

In addition, according to any one of the above-mentioned problem solving means, it is possible to suggest a neural network acceleration method and apparatus for reducing the amount of memory access data by varying quantization on data of a neural network.

The effects obtained in the disclosed embodiments are not limited to the effects mentioned above, and other effects not mentioned are obvious to those skilled in the art to which the embodiments disclosed from the following description belong It can be understood.

1 is a block diagram illustrating a neural network accelerator according to one embodiment.
2 is a flowchart illustrating a method of accelerating a neural network according to an exemplary embodiment of the present invention.
3 to 10 are reference views for explaining a neural network acceleration method according to an embodiment.
11 to 12 are experimental data on the performance of the neural network acceleration method according to an embodiment.

Various embodiments are described in detail below with reference to the accompanying drawings. The embodiments described below may be modified and implemented in various different forms. In order to more clearly describe the features of the embodiments, detailed descriptions of known matters to those skilled in the art are omitted. In the drawings, parts not relating to the description of the embodiments are omitted, and like parts are denoted by similar reference numerals throughout the specification.

Throughout the specification, when a configuration is referred to as being "connected" to another configuration, it includes not only a case of being directly connected, but also a case of being connected with another configuration in between. In addition, when a configuration is referred to as "including ", it means that other configurations may be included, as well as other configurations, as long as there is no specially contradicted description.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

Before describing this, we first define the meaning of the terms used below.

Hereinafter, the 'neural network' may include an input layer, at least one hidden layer, and an output layer, and each layer may include at least one 'node'. The node of each layer can form the connection relation with the node of the next layer.

The 'data' is information input or output to each layer of the neural network. It is a weight that determines the reflection intensity of data input to the layer when it is transmitted to the next layer, and activation, which is input / And the like.

'Quantization' is a technique that expresses data with a small number of bits while minimizing quality degradation as a result of the calculation.

This quantization method is a binary quantization method applying a binary weight, for example, and an N (> 1) bit quantization method applied to a network having a high complexity. Even in the case of a deep neural network, Clipping based quantization method that minimizes possible cross entropy and balanced quantization method which is 4 bit quantization method with accuracy loss within 3%.

Terms other than those defined above are explained below separately.

1 is a block diagram for explaining a neural network acceleration apparatus 10 according to an embodiment.

The neural network accelerator 10 may be implemented as a computer, a portable terminal, a television, a wearable device, or the like, which can be connected to a remote server through a network N or connected to other terminals and servers. Here, the computer includes, for example, a notebook computer, a desktop computer, a laptop computer, and the like, each of which is equipped with a web browser (WEB Browser), and the portable terminal may be a wireless communication device , Personal Communication System (PCS), Personal Digital Cellular (PDC), Personal Handyphone System (PHS), Personal Digital Assistant (PDA), Global System for Mobile communications (GSM), International Mobile Telecommunication (IMT) (W-CDMA), Wibro (Wireless Broadband Internet), Smart Phone, Mobile WiMAX (Mobile Worldwide Interoperability for Microwave Access) (Handheld) based wireless communication device. In addition, the television may include an Internet Protocol Television (IPTV), an Internet television (TV), a terrestrial TV, a cable TV, and the like. Further, the wearable device is an information processing device of a type that can be directly worn on a human body, for example, a watch, a glasses, an accessory, a garment, shoes, and the like. The wearable device can be connected to a remote server via a network, Lt; / RTI >

Referring to FIG. 1, the neural network acceleration apparatus 10 according to an embodiment may include an input / output unit 110, a control unit 120, a communication unit 130, and a memory 140.

The input / output unit 110 may include an input unit for receiving an input from a user, and an output unit for displaying information such as a result of a task or a state of the neural network accelerator 10. [ For example, the input / output unit 110 may include an operation panel for receiving user input and a display panel for displaying a screen.

In particular, the input unit may include devices capable of receiving various types of user input, such as a keyboard, a physical button, a touch screen, a camera or a microphone. Also, the output unit may include a display panel or a speaker. However, the present invention is not limited to this, and the input / output unit 110 may include various input / output support structures.

The control unit 120 controls the overall operation of the neural network device 10 and may include a processor such as a CPU. The control unit 120 may control other components included in the neural network device 10 to perform an operation corresponding to the user input received through the input /

For example, the control unit 120 may execute a program stored in the memory 140, read a file stored in the memory 140, or store a new file in the memory 140. [

The control unit 120 may perform quantization on data used in a node constituting each of at least one layer constituting the neural network by at least one method.

That is, the control unit 120 can apply the high-precision quantization method to a small number of outlier data based on the data value on the neural network, and separately apply the low-precision quantization method to the remaining normal data .

For example, the control unit 120 can obtain the distribution of the weight according to the value by counting the weight on the neural network based on the value of the weight, and divides the interval according to the number of counted weights by the weight value . At this time, the controller 120 can divide the interval narrower as the absolute value of the weight is closer to 0, and can divide the interval broader as the absolute value of the weight is larger than 0.

Alternatively, for example, when the number of weights is equal to or greater than a predetermined number based on the number of weights, the control section 120 may divide the weight value section by 0.05 intervals. If the number of weights is equal to or less than the preset number, You can divide the interval by interval.

The controller 120 may apply the quantization method to be applied to the data according to the characteristics of data included in each section for at least one section.

At this time, the quantization method applied by the control unit 120 can determine and apply a quantization level for data based on an error between the data and the quantized data.

For example, the control unit 120 can apply a high-precision quantization method in which the quantization level is 16 bits because the error between the quantization data and the quantized data is large in a section of 0.4 or more in which the value of the weight is relatively large, A low-precision quantization method with a quantization level of 8 bits can be applied to an interval in which the error value of the data and the quantized data is small and the weight value is less than 0.4.

In this way, even if the control unit 120 applies a quantization of different methods for each section based on the distribution of data, and thus has a long tail shape in which the distribution of the data spreads to the left and right by the outlier data having a large value , It is possible to reduce the degradation of the neural network output quality due to the quantization of data belonging to the long tail region while increasing the calculation speed by performing quantization with limited bits.

Quantization considering anomalies has two advantages over existing linear quantization in reducing quantization errors. First, most of the data has fewer quantization errors. Second, since the outliers are represented by high precision values, there is no quantization error in the outlier data. Since the outliers are large data, the quantization considering the data has a large effect on the result. Therefore, it reduces the overall quantization error, thereby allowing the data to be represented with a small number of bits compared to the existing linear quantization.

Thereafter, the control unit 120 classifies the data into the abnormal data and the normal data according to the data format, and stores the data in the buffer on the memory 140. The data stored in the buffer classifies the abnormal data and the normal data to perform the calculation .

According to the embodiment, the control unit 120 controls the operation of the control unit 120 based on the number of bits representing the quantized activation included in the data, Lt; RTI ID = 0.0 > normal activation. ≪ / RTI >

For example, the control unit 120 can determine whether the number of bits representing the activation exceeds 8 bits, and can classify the abnormal value activation to exceed 8 bits and the normal activation to 8 bits or less.

Then, the controller 120 can separately calculate the normal activation and the abnormal activation, and calculate them in parallel.

If the activation input to the node according to one embodiment is a normal activation, the control unit 120 may perform computation via a normal computation group that performs computations for normal activation.

For example, the controller 120 may transmit normal activation, which is 16 four-bit activation stored in the buffer of the memory 140, to the cluster activation buffer of the normal calculation group for calculating the normal activation, The weight can be moved to the cluster weight buffer of the normal calculation group.

In this case, the cluster weight buffer may be configured in the form of a table, and each row of the table includes an 8-bit pointer for expressing the ideal weight, which is an 8-bit weight, and a 4-bit pointer Bit index values. And the cluster activation buffer may consist of 16 rows of 4-bit activation normal activation in one row.

The controller 120 may calculate the weight stored in the cluster weight buffer and the activation stored in the cluster activation buffer in parallel. At this time, the controller 120 may schedule the calculation of the activation except for the zero-activation among the activation, which is data output from the node constituting the neural network.

For example, the control unit 120 may assign 16 weights from the cluster weight buffer and 16 activations from the cluster activation buffer to 16 calculation units constituting the normal calculation group, respectively. The control unit 120 may perform a convolution calculation between one 4-bit weight and one 4-bit activation in one cycle through the calculation unit. Thereafter, the control unit 120 may combine the calculated results in each of the 16 calculation units and store them in a buffer.

At this time, if there is an outlier weight among the weights stored in the cluster weight buffer, the control unit 120 can calculate the outlier weight and the activation in two cycles.

For example, if there is one 8-bit outlier weight among the 16 weights, the controller 120 performs 4-bit activation and 4-bit convolution calculations on the upper 4 bits (MSB) of the 8-bit outlier weight, And performing the activation and the 4-bit convolution computation on the remaining lower 4 bits and summing them, thereby performing the convolution computation of the ideal weight and the activation in two cycles.

If the activation input to the node according to another embodiment is an outlier activation, the controller 120 may perform computation through an outlier computation group that performs computations for outlier activation.

For example, the control unit 120 obtains 16 16-bit activations, which are the outlier activation stored in the buffer, and obtains 16 weights from the cluster weight buffer, assigns them to each of the 16 calculation units constituting the outlier calculation group, To perform the convolution calculation.

At this time, if one of the 16 weights is included, the control unit 120 performs the convolution calculation through the first calculation unit for the upper bit and the 16-bit activation of the outlier weight, The calculation can be performed in parallel in two calculation units by calculating 16-bit activation through the second calculation unit, respectively.

Thereafter, the control unit 120 may store the calculation result at the node in the buffer based on the result calculated in the normal calculation group and the calculation result in the outlier calculation group.

For example, the controller 120 accesses two buffers among the cluster output tri-buffers constituted by three buffers, and adds the new results calculated in the normal calculation group to the previously calculated results. have. Thereafter, the control unit 120 may calculate the final result by adding the result calculated in the outlier calculation group to the result stored in the cluster output tri-buffer.

In this way, in the node of the neural network that processes the calculation, the number of calculation using data corresponding to the long tail can be greatly reduced in the distribution of data by selectively changing the calculation group according to the quantization method for the data, And the resultant error of the neural network due to the quantization can be minimized.

The communication unit 130 can perform wired / wireless communication with another device or a network. To this end, the communication unit 130 may include a communication module supporting at least one of various wired / wireless communication methods. For example, the communication module may be implemented in the form of a chipset.

The wireless communication supported by the communication unit 130 may be Wi-Fi (Wireless Fidelity), Wi-Fi Direct, Bluetooth, UWB (Ultra Wide Band), NFC (Near Field Communication), or the like. The wired communication supported by the communication unit 130 may be, for example, USB or High Definition Multimedia Interface (HDMI).

Various types of data such as files, applications, programs, and the like can be installed and stored in the memory 140. [ The control unit 120 may access the data stored in the memory 140 and use the data or store the new data in the memory 140. [ Also, the control unit 120 may execute a program installed in the memory 140. [ Referring to FIG. 1, a program for performing a quantization method may be installed in the memory 140.

The memory 140 may store the calculated data in the normal calculation group or the outlier calculation group of the control unit 120. [

For this purpose, a buffer may be allocated in the memory 140, for example, a buffer for storing data, a cluster activation buffer for storing normal activation, a cluster weight buffer for storing weights, and a cluster An output tri buffer can be assigned.

That is, in the case of the neural network hardware accelerator (Cnvlutin), the multiplication operation for zero input activation is skipped to reduce the calculation amount. Existing hardware accelerators do not have a special structure to support quantization, and there is a structure to reduce the number of effective bits of data in calculation. However, it is assumed that the number of data bits is 32 bits or 16 bits in memory access.

Such a neural network accelerator 10 uses high-precision data for a small number of outliers and low-precision data for most of the remaining data, resulting in data of 4-bit level even in the case of a large-scale deep neural network It differs from conventional accelerators in that it allows representation and calculation.

2 is a flowchart illustrating a method of accelerating a neural network according to an exemplary embodiment of the present invention.

The neural network acceleration method according to the embodiment shown in FIG. 2 includes steps that are processed in a time-series manner in the neural network accelerator 10 shown in FIG. Therefore, even though omitted from the following description, the above description of the neural network accelerator 10 shown in FIG. 1 can be applied to the neural network acceleration method according to the embodiment shown in FIG.

First, the neural network accelerator 10 may perform quantization on data used in the neural network by at least one method (S2001).

According to an embodiment, the neural network accelerator 10 can acquire a distribution of data used in a neural network based on a range of values that the data may have, and perform the same quantization method based on the distribution of data You can divide the interval of the data value to be divided.

For example, if the weight value, which is the reflection strength of data between at least one node on the layer constituting the neural network, is determined according to the learning, the neural network accelerator 10 calculates the same value based on the value of the weight on the neural network The weights of the branches can be counted to obtain the distribution of the weight values. The neural network accelerating apparatus 10 sets the interval of the weight value to perform the same quantization method in inverse proportion to the number of weights and narrows the interval of the value of the weight for performing the same quantization method in a section in which the number of data is small Can be set.

3 is a table showing the setting of intervals in the distribution of weights. Referring to this, the neural network accelerator 10 can set the interval 301 between 0 and 0.1, which is a section having a large number of weights, and the interval 301 with a small number of weights is 0.1 to 0.4, (302).

The neural network accelerating apparatus 10 may apply the quantization method differently according to at least one section set based on the distribution of data and may calculate the level of the quantization to be applied to each section based on the error between the data and the data to which the quantization is applied Can be determined differently.

According to the embodiment, the neural network acceleration apparatus 10 can perform a high-precision quantization method on the outlier data, which is data belonging to a section exceeding a preset value, based on the data value belonging to the set interval, A low-precision quantization method can be performed for the normal data.

For example, the neural network accelerating apparatus 10 can perform a high-precision quantization method in which the number of bits is 16 bits for data with a large error due to quantization and a section with a relatively large weight value of 0.4 or more. When the data value is less than 0.4 A low-precision quantization method in which the number of bits is 8 bits can be applied to data having a small error due to quantization.

4 is a graph showing the distribution of values obtained by performing different quantization according to the weight distribution included in the data on the neural network acceleration apparatus 10. In FIG.

Referring to FIG. 4, the weight value on the first graph 401 is broadened by the ideal weight 402, 403 having a large absolute value of the prime number on the neural network. Accordingly, when the conventional linear quantization method is applied, much error occurs in the quantization as in the second graph 404.

On the other hand, the neural network accelerator 10 can classify the values of the outlier weights 402 and 403 and the remaining weights based on the value of the weight, as in the third graph 405, By applying a high-precision quantization method to the outlier weights 406 and 407, a quantization error with respect to the value of the outlier weight can be reduced. By applying the low-precision quantization to the normal weight, the weight can be expressed with a small number of bits have.

By performing different quantization for each data, data can be expressed with a smaller number of bits than in the case of performing uniform quantization (e.g., linear quantization or log quantization) on all data, and at the same time, quantization error The output quality of the neural network can be maintained.

Thereafter, the neural network acceleration apparatus 10 can store the quantized data in the buffer (S2002).

5 is a diagram illustrating the structure of a neural network accelerator 10 considering outlier data. Referring to FIG. 5, the neural network accelerator 10 may include a PE swarm 501, Includes a Swarm buffer 503 for storing information calculated in the PE cluster 502 and the PE cluster 502 and a Swarm controller 504 for controlling the operation of the PE cluster 502 and the communication with the main memory . Such a neural network accelerator 10 may store the data to be quantized and computed in the PE swarm buffer 503. At this time, the Swarm buffer 503 is used as a buffer for communication with the main memory, and can store the input and output values of the PE cluster 502. The PE cluster 502 may consist of PE groups, a cluster buffer, and a controller. The PE group can be composed of 17 calculation units each, and the calculation group can be divided into two types, a normal calculation group and an outlier calculation group. The calculation result of the calculation group can be stored in the cluster output tri-buffer.

Then, the neural network acceleration apparatus 10 can perform computation on data separately according to the representation format of the quantized data (S2003).

To this end, the neural network acceleration device 10 can perform the calculation differently based on the quantization format of the inputted data, by classifying it into the abnormal value activation and the normal activation.

For example, the neural network accelerating apparatus 10 may classify as normal activation if the bit is less than 16 bits based on the bit of the activation obtained from the buffer, and classify it as abnormal value activation if the bit is 16 bits.

Then, the neural network acceleration device 10 can be separately calculated by the normal activation and the abnormal activation.

At this time, the neural network acceleration apparatus 10 can improve the performance by raising the operation rate of the calculation group through the scheduling for the calculation group for calculating the data.

6 is an exemplary diagram illustrating scheduling computation groups. The neural network accelerator 10 acquires 1X1X16 activation through a computation group to perform computation and generates 1X1X16 activation as an output. The neural network accelerator 10 sends a new 1X1X16 activation 602 to the first computation group 602 if the first computation group 601 completes calculations according to the number of non-zero activations to perform only non-zero computations 603) so as to continue the calculation.

If normal activation is input according to one embodiment, the neural network accelerator 10 may store the normal activation and the weight in a buffer and calculate through the normal calculation group.

7 is an exemplary diagram showing the internal operation of the PE cluster of the neural network accelerator 10. [ Referring to this, the neural network accelerator 10 can acquire and store 16 4-bit weights for each row from a buffer in a cluster weight buffer 701 configured in a tabular form. The neural network accelerator 10 may also acquire and store 16 4-bit activations from the buffer into the cluster activation buffer 702.

The neural network acceleration device 10 may then perform convolution calculations on the normal activation and weight stored in the buffer.

For example, the neural network accelerator 10 may perform four cycles of 4-bit activation and 4-bit weight convolution computation, which are normally activated in each of the plurality of normal calculation groups, in one cycle.

At this time, if the weight at which the 4-bit activation and the convolution calculation are performed is the odd value weight of 8 bits, the neural network acceleration device 10 performs the convolution calculation between the upper 4 bits and the 4-bit activation of the ideal value weight, The final result value can be obtained by adding the calculated result after performing the convolution calculation between the lower 4 bits and the 4-bit activation.

FIG. 8 is an exemplary diagram showing operations in one calculation group. FIG. With reference to this, the neural network accelerator 10 can provide 16 normal activations and 16 weights obtained to each of the 16 normal calculation units 802 for calculating the normal activation in the calculation group, Lt; RTI ID = 0.0 > normal-activation < / RTI >

At this time, when the outlier weight among the 16 weights is included, the neural network accelerator 10 allocates the upper 4 bits of the outlier weight to the outlier calculation unit 801 in the calculation group to perform the normal activation and the convolution calculation, The lower 4 bits of the ideal value weight can be assigned to the normal calculation unit 802 to perform the normal activation and the convolution calculation. Thereafter, the neural network acceleration apparatus 10 can calculate the final result in two cycles by summing the result calculated in the outlier calculation unit and the result calculated in the normal calculation unit.

The neural network accelerator 10 may then store the computed results in a cluster output tri-buffer.

According to another embodiment, when an anomaly activation is input, the neural network accelerator 10 may calculate an outlier activation and a weight through an outlier calculation group.

For example, the neural network accelerator 10 may calculate the 16-bit activation and the weight, which are the outlier activation, using a full-precision calculation unit.

Fig. 9 is an exemplary diagram showing a calculation group for calculating an outlier activation. Fig. Referring to this, the neural network accelerator 10 may store the outlier activation stored in the PE swarm buffer as a tabular form 901 in the anomalous activation FIFO, and may use the weight input from the cluster weight buffer to generate 16 full precision computation units Can be used to perform convolution calculations.

At this time, when the ideal value weight is input, the upper 4 bits of the ideal value weight and the abnormal value activation are calculated by the calculation result of the abnormal value full precision calculation unit, the lower 4 bits of the ideal value weight and the abnormal value activation are set as the normal full precision The final result can be calculated by summing the calculation results through the calculation unit.

The neural network accelerator 10 may then store the result to be computed in a cluster output tri-buffer.

Thereafter, the neural network accelerator 10 may calculate the final result based on the results calculated by the normal calculation group and the outlier calculation group, respectively.

For example, the neural network accelerator 10 may sum the results stored in the cluster output tri-buffer to obtain the final result.

10 is an exemplary diagram illustrating a process of storing data in a cluster output tri-buffer to calculate a final result. Referring to this, the neural network accelerator 10 can access two buffers in a normal calculation group and store a result of adding a new second result 1002 to an existing stored first result 1001, May be added to the second result 1002 to calculate the final result 1003.

In this way, the quantization method for the data on the neural network is performed in consideration of the outlier data, and the computed amount is reduced while maintaining the quality of the output result with a small number of bits by separately calculating the quantized data according to the number of bits of data The amount of data written to memory can be reduced.

11 to 12 are experimental results using a neural network acceleration method. Referring to FIG. 11, the present invention is applied to VGG-16, which is a representative example of a deep neural network, and the results are compared with existing hardware accelerators. As shown in the figure, the 4-bit accelerator (8-bit outlier, OLAccel8) of the present invention is faster than the 8-bit version of Eyeriss and ZeNA by more than 30% and can reduce energy more than 2 times.

 Referring to FIG. 12, there is shown a result of comparing ResNet-18 with an existing hardware accelerator for ImageNet processing. In this case, the ZeNA results were not found in the existing paper and only Eyersiss was compared. The 4 bit accelerator (8 bit outlier value, OLAccel8) of the present invention compared with the Eyeriss 8 bit has a performance four times or more and an energy reduction of 40 percent or more.

The present invention is a technique for reducing data size (number of bits) in a deep neural network and can be used for performance and learning of all neural networks. Therefore, it is applicable to all fields where deep running is applied.

The term " part " used in the above embodiments means a hardware component such as a software or a field programmable gate array (FPGA) or an ASIC, and the 'part' performs certain roles. However, 'part' is not meant to be limited to software or hardware. &Quot; to " may be configured to reside on an addressable storage medium and may be configured to play one or more processors. Thus, by way of example, 'parts' may refer to components such as software components, object-oriented software components, class components and task components, and processes, functions, , Subroutines, segments of program patent code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables.

The functions provided within the components and components may be combined with a smaller number of components and components or separated from additional components and components.

In addition, the components and components may be implemented to play back one or more CPUs in a device or a secure multimedia card.

The neural network acceleration method according to the embodiment described with reference to FIG. 2 may also be implemented in the form of a computer-readable medium for storing instructions and data executable by a computer. At this time, the command and data may be stored in the form of program code, and when executed by the processor, a predetermined program module may be generated to perform a predetermined operation. In addition, the computer-readable medium can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. The computer-readable medium can also be a computer storage medium, which can be volatile and non-volatile, implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, For example, computer recording media may include magnetic storage media such as HDDs and SSDs, optical recording media such as CD, DVD and Blu-ray discs, or other types of media accessible via a network. May be the memory included in the server.

The neural network acceleration method according to the embodiment described with reference to FIG. 2 may also be implemented as a computer program (or a computer program product) including instructions executable by a computer. A computer program includes programmable machine instructions that are processed by a processor and can be implemented in a high-level programming language, an object-oriented programming language, an assembly language, or a machine language . The computer program may also be recorded on a computer readable recording medium of a type (e.g., memory, hard disk, magnetic / optical medium or solid-state drive).

Therefore, the neural network acceleration method according to the embodiment described with reference to FIG. 2 can be implemented by a computer program as described above being executed by the computing device. The computing device may include a processor, a memory, a storage device, a high-speed interface connected to the memory and a high-speed expansion port, and a low-speed interface connected to the low-speed bus and the storage device. Each of these components is connected to each other using a variety of buses and can be mounted on a common motherboard or mounted in any other suitable manner.

Where the processor may process instructions within the computing device, such as to display graphical information to provide a graphical user interface (GUI) on an external input, output device, such as a display connected to a high speed interface And commands stored in memory or storage devices. As another example, multiple processors and / or multiple busses may be used with multiple memory and memory types as appropriate. The processor may also be implemented as a chipset comprised of chips comprising multiple independent analog and / or digital processors.

The memory also stores information within the computing device. In one example, the memory may comprise volatile memory units or a collection thereof. In another example, the memory may be comprised of non-volatile memory units or a collection thereof. The memory may also be another type of computer readable medium such as, for example, a magnetic or optical disk.

And the storage device can provide a large amount of storage space to the computing device. The storage device may be a computer readable medium or a configuration including such a medium and may include, for example, devices in a SAN (Storage Area Network) or other configurations, and may be a floppy disk device, a hard disk device, Or a tape device, flash memory, or other similar semiconductor memory device or device array.

It will be apparent to those skilled in the art that the above-described embodiments are for illustrative purposes only and that those skilled in the art will readily understand that other embodiments can be readily modified without departing from the spirit or essential characteristics of the embodiments described above You will understand. It is therefore to be understood that the above-described embodiments are to be considered in all respects only as illustrative and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

It is to be understood that the scope of the present invention is defined by the appended claims rather than the foregoing description and should be construed as including all changes and modifications that come within the meaning and range of equivalency of the claims, .

10: Neural network accelerator
110: Input / output unit
120:
130:
140: Memory

Claims (14)

An apparatus for accelerating data processing on a neural network,
A controller for performing quantization on the data based on characteristics of data on the neural network and separately performing calculations on the data according to a representation format of the quantized data; And
And a memory for storing the quantized data. The method according to claim 1,
Wherein,
And determines a quantization method to apply to the data based on a distribution of data on the neural network. 3. The method of claim 2,
Wherein,
And determines a quantization level for the data based on an error between the data and the quantized data. 3. The method of claim 2,
The data includes:
The abnormal value data having the data value exceeding a predetermined value and the normal data having the data value equal to or less than a predetermined value,
Wherein,
And separates each of the abnormal value data and the normal data to perform the calculation. 5. The method of claim 4,
Wherein,
Wherein activation is scheduling a calculation for activation other than zero-activation among activation, which is data output from a node constituting the neural network. 5. The method of claim 4,
Wherein,
Wherein the calculation is performed by dividing the outlier weight among the weights, which is data representing the reflection strength of the activation, into a predetermined number of upper bits and remaining bits, to a node constituting the neural network. A method for accelerating data processing on a neural network,
Performing quantization on the data based on characteristics of the data on the neural network;
Storing the quantized data; And
And separately performing calculations on the data according to a representation format of the quantized data. 8. The method of claim 7,
Wherein the step of performing quantization on the data comprises:
Determining a quantization method to apply to the data based on a distribution of data on the neural network. 9. The method of claim 8,
Wherein the step of determining the quantization method comprises:
Determining a quantization level for the data based on an error between the data and the quantized data. 9. The method of claim 8,
The data includes:
The abnormal value data having the data value exceeding a predetermined value and the normal data having the data value equal to or less than a predetermined value,
Separately performing the calculations on the data comprises:
And separating each of the outlier data and the normal data to perform a calculation. 11. The method of claim 10,
Wherein the step of separating the abnormal data and the normal data to perform the calculation comprises:
And scheduling a calculation for activation other than zero-activation among activation, which is data output from a node constituting the neural network. 11. The method of claim 10,
The neural network acceleration method includes:
Further comprising the step of dividing the outlier weight among the weights, which is data representing the reflection intensity of the activation, into a predetermined number of upper bits and remaining bits, Neural network acceleration method. A computer-readable recording medium on which a program for carrying out the method according to claim 7 is recorded. A computer program stored in a medium for performing the method of claim 7 performed by a neural network acceleration device.

Images