KR102383962B1 - Deep learning accelerator with variable data encoder/decoder - Google Patents

Abstract

Provided is a deep learning accelerator comprising a variable data compressor/decompressor. A deep learning accelerator, according to an embodiment of the present invention, comprises: an RDMA which directly accesses an external memory and reads data for a deep learning operation from the external memory; a decompressor which decompresses the data read by the RDMA; an input buffer in which the data decompressed by the decompressor is stored; an operator which performs the deep learning operation by means of the data stored in the input buffer; and a controller which identifies the statuses of the RDMA and the input buffer and controls decompression of the decompressor on the basis of the identified statuses. The compressor/decompressor can be controlled according to the statuses of a DMA and a buffer, thereby maintaining the output of the deep learning accelerator at a maximum level.

Description

Deep learning accelerator with variable data encoder/decoder}

The present invention relates to image processing and SoC (System on Chip) technology, and more particularly, to a deep learning accelerator device for controlling a compressor/decompressor by grasping a hardware state for data supply of deep learning hardware.

Most of the conventional deep learning hardware accelerators receive input data (feature map) and input convolution parameters (weight), and aim to perform calculations quickly.

When accessing an external memory, it cannot exceed the allowable bandwidth of the external memory, which is a physical constraint, so a lot of data can be supplied when data input/output is compressed.

However, even if a simple lossless compression is used as compression/restore and a technique having a high compression ratio is used, there is a problem in that the size of the hardware for compression/restore becomes larger than that of the accelerator when real hardware is implemented.

In addition, when lossy compression is used, a larger problem occurs because it is a method in which performance degradation occurs.

The present invention has been devised to solve the above problems, and an object of the present invention is to continuously monitor the state of the accelerator and to produce the maximum output. Compression/restore according to the DMA and buffer states It is to provide a deep learning accelerator that controls the machine.

According to an embodiment of the present invention for achieving the above object, a deep learning acceleration device includes: RDMA (Read Direct Memory Access) for directly accessing an external memory, and reading data for a deep learning operation from the external memory; a restorer that restores data read from RDMA; an input buffer in which data restored by the restorer is stored; an operator that performs a deep learning operation with data stored in the input buffer; and a controller for determining the RDMA and the input buffer conditions, and controlling the restoration operation of the restorer based on the identified conditions.

Deep learning acceleration apparatus according to an embodiment of the present invention, an output buffer in which data calculated by deep learning in the calculator is stored; a compressor for compressing data stored in the output buffer; WDMA (Write Direct Memory Access) that directly accesses the external memory and writes the compressed data in the compressor to the external memory; further comprising, the checker understands the status of the WDMA and the output buffer, and based on the identified situation It is possible to control the compression operation of the compressor.

The checker can grasp the bandwidth situation of RDMA and WDMA. In addition, when the identified RDMA bandwidth situation is not good, the checker can control the restorer to restore data applied from the RDMA in advance and store it in the internal cache.

And, when the identified WDMA bandwidth situation is not good, the checker may control the compressor to pre-compress the data stored in the output buffer and store it in the internal cache.

The decompressor and the compressor can minimize parallel processing when there are many zeros in the IFmap and OFmap data more than a threshold.

The restorer and the compressor can maximize parallel processing when the number of 0 in the weight data is less than or equal to a threshold.

On the other hand, according to another embodiment of the present invention, a deep learning acceleration method, RDMA (Read Direct Memory Access), directly accessing an external memory, reading data for a deep learning operation from the external memory; restoring, by a restorer, data read from RDMA; storing, by the input buffer, the restored data; performing, by the calculator, a deep learning operation on the stored data; and, by the checker, determining the RDMA and the input buffer conditions, and controlling the restoration operation of the restorer based on the identified conditions.

As described above, according to the embodiments of the present invention, it is possible to control the compressor/decompressor according to the state of the DMA and the buffer, so that it is possible to maintain the output of the deep learning accelerator to the maximum. In particular, it is possible to control parallel data processing and speed as needed, and compression/restore compression/restore by applying internal/external memory access pattern change for data processing by channel/weight.

1 is a view showing a deep learning acceleration device to which the present invention is applicable;
2 is a block diagram showing the structure of a deep learning acceleration apparatus according to an embodiment of the present invention;
Figure 3 is a block diagram showing the detailed structure of the operator shown in Figure 2;
4 is an example of a restorer supporting the Run mode,
5 and 6, the parallel processing process in the Index Union Module and the Data Collect Module of the restorer illustrated in FIG. 4, and,
7 is a flowchart provided to explain a method for controlling an operation of a deep learning acceleration apparatus according to another embodiment of the present invention.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

1 is a diagram illustrating a deep learning acceleration device to which the present invention is applicable. The illustrated deep learning accelerator consists of a read direct memory access (RDMA) (11), an input buffer (12), an operator (13), an output buffer (14), and a write direct memory access (WDMA) (15). do.

The deep learning accelerator receives data from the external memory 10 , performs a deep learning operation, and outputs and stores the operation result to the external memory 10 .

The data input from the external memory 10 is IFmap (Input Feature map: feature data of the input image) and Weight (convolution parameter of the deep learning model), and the deep learning operation result output to the external memory 10 is OFmap ( output feature map).

The operator 13 performs a deep learning operation with data stored in the input buffer 12 . In this process, the operator 13 sequentially calculates and sums each channel for filtering, applies simultaneous filtering to multiple channels, and may simultaneously process images of multiple channels with filters.

As described above, the IFmap and Weight are input from the external memory 10 and the OFmap must be stored in the external memory 10 . For low-power operation, data transmission/reception with the external memory 10 should be reduced.

I/O bandwidth for sufficient data input/output is very important. In general, the amount of data is reduced through compression/restore of input/output data during operation, but in a specific layer, the data compression rate becomes worse.

In addition, when the external memory 10 cannot be accessed due to the sharing of the external memory 10, a data shortage may occur, which may cause malfunction of the compressor/decompressor. To this end, it is necessary to control the compressor/decompressor according to the state of the DMA and the buffer in order to sufficiently provide data.

Accordingly, in an embodiment of the present invention, a light-weight compressor/decompressor is applied to the deep learning acceleration device, but a technique for controlling the compressor/decompressor by checking the state of the DMA and the buffer is proposed.

2 is a block diagram illustrating the structure of a deep learning acceleration apparatus according to an embodiment of the present invention. Deep learning acceleration apparatus according to an embodiment of the present invention, as shown in FIG. 2 , RDMA 110 , a decoder 120 , an input buffer 130 , an operator 140 , an output buffer 150 . ), a compressor (Encoder) 160, WDMA (170) and a checker (Checker) (180) is configured to include.

The RDMA 110 directly accesses the external memory 10 and reads data for a deep learning operation from the external memory 10 . Data read by the RDMA 110 are IFmap and Weight.

The decompressor 120 restores (decompresses) the compressed data (IFmap and weight) read by the RDMA 110 . Data restored by the restorer 120 is stored in the input buffer 130 .

The calculator 140 is a module for performing a deep learning operation with data stored in the input buffer 130 . FIG. 3 is a block diagram illustrating a detailed structure of the calculator 140 shown in FIG. 2 .

As shown, the operator 140 includes a convolution operation module 141, an address tree module 142, a batch normalization module 143, an Add Bias module 144, and an Activation module 145 necessary for a deep learning operation. and a Maxpool module 146 .

The output buffer 150 is a buffer in which OFmap, which is data calculated by deep learning by the operator 140, is stored. The compressor 160 compresses the data OFmap stored in the output buffer 150 . The WDMA 170 directly accesses the external memory 10 and stores data compressed by the compressor 160 in the external memory 10 .

The checker 180 checks the RDMA 110 and the input buffer 130 and checks the output buffer 150 and the WDMA 170, to understand the bandwidth situation of the RDMA 110 and the bandwidth situation of the WDMA 170 .

And, the checker 180 controls the restoration/compression operation of the restorer 120/compressor 160 based on the DMA bandwidth situation (channel situation).

Specifically, when the bandwidth situation of the RDMA 110 is not good, the checker 180 restores the data applied from the RDMA 110 in advance by the restorer 120 and stores it in the internal cache.

In addition, when the bandwidth situation of the WDMA 110 is not good, the checker 180 allows the compressor 160 to pre-compress the data stored in the output buffer 150 and store it in the internal cache.

In the case of a compressor/decompressor that supports run mode, it consists of bits (NZ) allocated as data allocation (RI) data for the number of runs specified by the user. That is, when the compressed stream is input as RI bits + NZ bits and data is processed, it means that at least one data of up to 2^RI pieces of data can be generated.

In the case of N parallel processing, a maximum of N*(2^RI) can be processed, so an additional memory that stores the information on the valid stream of the initial data in the internal core and stores the location of the required data in need.

Since it is possible to secure enough data required by the core with a few bits of internal memory, if the memory access channel is bad, parallel processing can be performed in advance to secure the necessary data in advance (depending on the size of the internal global buffer). act accordingly).

4 is a diagram illustrating the internal structure of the restorer. Data that needs to be stored include: 1) the number of current input data counts, 2) undecrypted input data, 3) data remaining in cache, 4) number of data remaining in cache, 5) target count to be output, 6) The number of currently output data counts is included. Signals received include: 1) 128-bit data & valid signal, 2) DMA ready signal, 3) target count to be output, 4) signal to save current state, 5) signal to call saved state, 6) state and This includes a signal to empty the buffer, and 7) a signal to request data coming to the cache. Output signals include: 1) 64-bit data & valid signal, 2) DMA data request signal, 3) current input data count number, 4) current output data count number, 5) save process in progress indication, 6) load process In progress indication, 7) emptying process in progress indication, and 8) resting status indication after process are included.

5 and 6 are diagrams illustrating a parallel processing process in the Index Union Module and Data Collect Module of the restorer illustrated in FIG. 4 , specifically, a process of supplying data for parallel processing required from input data and collecting output data. .

This example is when the user designates run as 3 bits and data as 8 bits and processes 8 in parallel. In the case of hardware, since it does not include complex operations, multiple parallel cores can be prepared and data compression/restore processing can be performed in parallel according to the DMA and buffer conditions.

In addition, in the case of activation (IFmap/OFmap), parallel processing is minimized to enable low-power operation by minimizing parallel processing in the case of layer data in which zeros occur more than the threshold, and in the case of weight, in the case of channel data with few zeros below the threshold Parallel processing is performed to the maximum so that the required data can be sufficiently secured.

7 is a flowchart provided to explain a method for controlling an operation of a deep learning acceleration apparatus according to another embodiment of the present invention.

As shown, the RDMA 110 of the deep learning accelerator reads data for a deep learning operation from the external memory 10 (S210). Then, the restorer 120 restores the compressed data read in step S210 and stores it in the input buffer 130 (S220).

The calculator 140 performs a deep learning operation with the data stored in step S220 and stores the deep learning operation result in the output buffer 150 (S230).

Then, the compressor 160 compresses the data stored in step S230 ( S240 ), and the WDMA 170 stores the data compressed in step S240 in the external memory 10 ( S250 ).

While steps S210 to S250 are performed, the checker 180 checks the RDMA 110 and the input buffer 130 and checks the output buffer 150 and the WDMA 170 to determine the DMA bandwidth situation (S260) ).

Then, the checker 180 controls the restoration/compression operation of the restorer 120/compressor 160 based on the DMA bandwidth situation identified in step S260 (S270).

So far, for a deep learning acceleration device including a variable data compressor / decompressor, a preferred embodiment has been described in detail.

In the above embodiment, a model capable of outputting the maximum output while continuously monitoring the state of the accelerator is presented, which is a model applicable to flexible deep learning devices and various networks and layers.

It is a hardware structure that allows parallel data processing and speed control as needed, and a compression/restore hardware structure that applies a change in internal/external memory access pattern for data processing by channel/weight for compression/restore in the deep learning accelerator.

On the other hand, it goes without saying that the technical idea of the present invention can be applied to a computer-readable recording medium containing a computer program for performing the functions of the apparatus and method according to the present embodiment. In addition, the technical ideas according to various embodiments of the present invention may be implemented in the form of computer-readable codes recorded on a computer-readable recording medium. The computer-readable recording medium may be any data storage device readable by the computer and capable of storing data. For example, the computer-readable recording medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical disk, hard disk drive, or the like. In addition, the computer-readable code or program stored in the computer-readable recording medium may be transmitted through a network connected between computers.

In addition, although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention as claimed in the claims In addition, various modifications are possible by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

110 : RDMA (Read Direct Memory Access)
120: restorer
130: input buffer
140: operator
141: convolution operation module
142: address tree module
143: batch normalization module
144 : Add Bias module
145: Activation module
146 : Maxpool module
150 : output buffer
160: compressor
170: WDMA (Write Direct Memory Access)
180: Checker

Claims (8)

RDMA (Read Direct Memory Access) that directly accesses external memory and reads data for deep learning operation from external memory;
a restorer that restores data read from RDMA;
an input buffer in which data restored by the restorer is stored;
an operator that performs a deep learning operation with data stored in the input buffer;
an output buffer in which data calculated by deep learning in the calculator is stored;
a compressor for compressing data stored in the output buffer;
Write Direct Memory Access (WDMA), which directly accesses the external memory and writes the compressed data in the compressor to the external memory;
A controller for controlling the restoration operation of the restorer based on the identified situation by grasping the RDMA and input buffer conditions, and controlling the compression operation of the compressor based on the grasped condition by identifying the WDMA and output buffer conditions; includes; do,
restorer and compressor,
Deep learning acceleration device characterized in that parallel processing is minimized when there are many zeros in IFmap and OFmap data more than a threshold.
delete The method according to claim 1,
The controller is
Deep learning accelerator, characterized in that it grasps the bandwidth situation of RDMA and WDMA.
4. The method according to claim 3,
The controller is
If the identified RDMA bandwidth situation is not good, a deep learning accelerator characterized in that the restorer restores the data applied from the RDMA in advance and controls it to be stored in the internal cache.
5. The method according to claim 4,
The controller is
If the identified WDMA bandwidth situation is not good, the deep learning accelerator, characterized in that the compressor controls the data stored in the output buffer to be pre-compressed and stored in the internal cache.
delete The method according to claim 1,
restorer and compressor,
A deep learning accelerator, characterized in that when the weight data has a small number of 0 below the threshold, the parallel processing is maximized.
RDMA (Read Direct Memory Access), by directly accessing an external memory, reading data for deep learning operation from the external memory;
restoring, by a restorer, data read from RDMA;
storing, by the input buffer, the restored data;
performing, by an operator, a deep learning operation with data stored in an input buffer;
The output buffer, storing the deep learning calculated data;
compressing, by the compressor, the data stored in the output buffer;
WDMA (Write Direct Memory Access), accessing the external memory directly, and writing the compressed data to the external memory;
controlling, by the controller, the conditions of the RDMA and the input buffer, and controlling the restoration operation of the restorer based on the identified conditions;
Including, by the controller, determining the status of the WDMA and the output buffer, and controlling the compression operation of the compressor based on the identified situation;
restorer and compressor,
Deep learning acceleration method characterized in that parallel processing is minimized when there are many zeros in IFmap and OFmap data more than a threshold.

Images