KR20220149281A - Apparatus and method for compressing feature map - Google Patents

Abstract

A method for compressing a feature map performed by a compressor according to one embodiment includes: acquiring a feature map from an input image based on one or more layers included in a convolution neural network (CNN) model; dividing the acquired feature map into unit tiles having a predetermined size and sequentially inputting the divided unit tiles into a register; generating first data indicating the number of zero tiles input to the register before a non-zero tile is input to the register, second data indicating a pixel having a zero value and a pixel having a non-zero value within the non-zero tile, and third data based on an activation value of a pixel having the non-zero value within the non-zero tile; and storing the first data, the second data, and the third data in a buffer.

Description

Apparatus and method for compressing feature maps

Hereinafter, techniques for an apparatus and method for compressing a feature map obtained based on a layer of a convolutional neural network model are provided.

The convolutional neural network architecture requires access to a large amount of off-chip memory (eg, dynamic random access memory (DRAM)), and the off-chip memory stores the parameters in an internal buffer through access to Access to these external DRAM accounts for the majority of network energy consumption. In general, a convolutional neural network accelerator device uses a method of reducing power consumption by compressing an output feature map of a layer included in a CNN model. However, the conventional feature map compression method has a disadvantage that the compression ratio decreases when there are not many zero ('0') values in the feature map. Hereinafter, in order to efficiently increase the compression ratio of the feature map, a method of compressing the feature map using the distribution of non-zero activation values and spatial correlation in the feature map explain about

A method of compressing a feature map performed by a compressor according to an embodiment includes a feature map based on one or more layers included in a convolutional neural network (CNN) model from an input image. obtaining, dividing the obtained feature map into unit tiles of a predetermined size, and sequentially inputting the divided unit tiles into a register; a non-zero tile is input to the register first data indicating the number of zero tiles input to the register before the number of zero tiles, second data indicating a pixel having a zero value and a pixel having a non-zero value within the non-zero tile, and the non-zero tile generating third data based on an activation value of a pixel having a non-zero value in a tile, and storing the first data, the second data, and the third data in the buffer may include steps.

The generating of the data may include indicating the number of zero tiles input to the register before the non-zero tiles are input to the register when all pixels within the unit tile input to the register have zero values. It may include increasing the count of the first data.

In the method of compressing a feature map performed by a compressor according to an embodiment, the count of first data indicating the number of zero tiles input to the register before the non-zero tiles are input to the register is critical The method may further include generating a threshold value as the first data and generating a zero value as the second data when the value is reached.

The generating of the data may include generating the third data by representing an active value of a pixel having a non-zero value in the non-zero tile as data having a first bit width. have.

The generating of the data may include classifying active values of pixels in the non-zero tile into one of an outlier, a non-outlier, and a zero value, and assigning the active value classified as an outlier to a first bit and generating the third data by representing an active value that is represented by data of a width and classified as a non-outlier as data of a second bit width smaller than the first bit width.

The step of inputting the divided unit tiles into the register includes dividing an active value of each pixel included in the unit tile into high-order bit data and low-order bit data and inputting them into the register, and generating the data includes: , determining whether higher bit data and lower bit data corresponding to the active value of the pixel are zero values, and classifying the active values into one of an outlier, a non-outlier, and a zero value.

The step of classifying the active value includes, when both the upper bit data and the lower bit data are non-zero values, classifying the active value as an outlier, the upper bit data is a zero value, and the lower bit data is a non-zero value classifying the active value as a non-outlier in the case of the case, and classifying the active value as a zero value when both high-order bit data and low-order bit data are zero values.

The generating of the data may include mapping a first value to a pixel having a zero value in the non-zero tile, mapping a second value to a pixel having a non-outlier, and mapping a second value to a pixel having an outlier. and generating the second data by mapping three values, wherein the first value, the second value, and the third value may be different data.

In the compressor for compressing the feature map according to an embodiment, the feature map has a predetermined size with respect to a feature map obtained based on one or more layers included in the convolutional neural network model from the input image. A register divided into unit tiles, to which the divided unit tiles are sequentially input, a comparator for comparing data stored in the register, and a non-zero tile inputted to the register until a non-zero tile is inputted to the register based on the result of the comparison First data indicating the number of zero tiles, second data indicating pixels having a zero value and pixels having a non-zero value in the non-zero tile, and non-zero in the non-zero tile a controller that generates third data based on an active value of a pixel having a value, and stores the first data, the second data, and the third data, and outputs the stored data to a dynamic random access memory (DRAM) It may contain a buffer to

When all pixels of the unit tile input to the register have a zero value, the controller is configured to count first data indicating the number of zero tiles input to the register before the non-zero tile is input to the register. can increase

The controller generates a threshold value as the first data when the count of first data indicating the number of zero tiles input to the register reaches a threshold value before the non-zero tiles are input to the register. and a zero value may be generated as the second data.

The controller may generate the third data by representing an active value of a pixel having a non-zero value in the non-zero tile as data having a first bit width.

The controller classifies an active value of pixels in the non-zero tile into one of an outlier, a non-outlier, and a zero value, and represents the active value classified as an outlier as data of a first bit width; The third data may be generated by representing an active value classified as a non-outlier as data having a second bit width smaller than the first bit width.

The register receives an input in which an active value of each pixel included in a unit tile is divided into high-order bit data and low-order bit data, and the comparator determines that the high-order bit data and low-order bit data corresponding to the active value of the pixel are zero. It is determined whether the value is a value, and the controller may classify the active value into one of an outlier, a non-outlier, and a zero value based on a result of the determination.

The controller, based on the comparison result, classifies the active value as an outlier when both the high-order bit data and the low-order bit data are non-zero values, the high-order bit data is a zero value, and the low-order bit data is a non-zero value In the case of a value, the active value is classified as a non-outlier, and when both the upper bit data and the lower bit data are zero values, the active value may be classified as a zero value.

The controller maps a first value to a pixel having a zero value within the non-zero tile, a second value maps a pixel having a non-outlier, and a third value to a pixel having an outlier to generate the second data, and the first value, the second value, and the third value may be different data.

A neural network accelerator device according to an embodiment uses a processing element array (PE array) that performs a convolutional neural network operation, and a feature map extracted from an input image based on one or more layers included in a convolutional neuron network model a buffer for obtaining , a controller that divides the acquired feature map into unit tiles of a predetermined size, and sequentially inputs the divided unit tiles to a compressor, and a non-zero tile until the non-zero tile is input to the register First data indicating the number of zero tiles input to the register, second data indicating pixels having a zero value and pixels having a non-zero value in the non-zero tile, and in the non-zero tile generate third data based on an active value of a pixel having a non-zero value in , store the first data, the second data, and the third data, and store the stored data in a dynamic random access memory (DRAM) It may include a compressor that outputs as

A method of decompressing a feature map performed by a decompressor according to an embodiment includes: inputting data in which a feature map obtained based on a layer of a convolutional network model is compressed into a register; generating and transmitting a corresponding number of zero tiles to a buffer, generating and transmitting non-zero tiles based on second data and third data to a buffer, wherein generating the non-zero tiles to a buffer In the transmitting step, a pixel having a zero value and a pixel having a non-zero value are distinguished from a pixel having a non-zero value in the non-zero tile by using the second data, and an active value of the pixel having a zero value corresponds to the zero value. and generating the non-zero tile by sequentially matching an active value of a pixel having a non-zero value with the third data.

A decompressor for decompressing a feature map according to an embodiment includes a register to which compressed data of a feature map obtained based on a layer of a convolutional network model is input, and a number of zero tiles corresponding to the first data. and a controller to generate and transmit to a buffer, and to generate and transmit a non-zero tile based on the second data and the third data to the buffer, wherein the controller is configured to generate a non-zero tile based on the second data and the third data, wherein the controller uses the second data to distinguishes a pixel having a zero value from a pixel having a non-zero value, an active value of the pixel having a zero value corresponds to a zero value, and an active value of the pixel having a non-zero value is the third data and The non-zero tiles may be generated by sequentially matching them.

1 illustrates a feature map in a CNN model and a distribution of active values of pixels in the feature map.
2 is a graph of spatial correlation between activation values within a specific distance within a feature map.
3 is a flowchart illustrating an operation of a neural network accelerator device according to an embodiment.
4 illustrates a method of compressing a feature map by an accelerator device according to an embodiment.
5 illustrates a configuration of an accelerator device according to an exemplary embodiment.
6 illustrates a structure of a compressor according to an embodiment.
7 illustrates a structure of a decompressor for decompressing a feature map according to an embodiment.
8 shows a graph of the reciprocal of normalized DRAM access according to the size of a unit tile.
9 is a graph showing a compression ratio and sparsity of a feature map according to various compression methods.
Figure 10 shows the values of normalized DRAM accesses of the compressed feature map with respect to quantized activation values.
11 shows a graph of energy consumed to store and load a feature map.

Specific structural or functional descriptions of the embodiments are disclosed for purposes of illustration only, and may be changed and implemented in various forms. Accordingly, the actual implementation form is not limited to the specific embodiments disclosed, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical spirit described in the embodiments.

Although terms such as first or second may be used to describe various elements, these terms should be interpreted only for the purpose of distinguishing one element from another. For example, a first component may be termed a second component, and similarly, a second component may also be termed a first component.

When a component is referred to as being “connected” to another component, it may be directly connected or connected to the other component, but it should be understood that another component may exist in between.

The singular expression includes the plural expression unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to designate that the described feature, number, step, operation, component, part, or combination thereof exists, and includes one or more other features or numbers, It should be understood that the possibility of the presence or addition of steps, operations, components, parts or combinations thereof is not precluded in advance.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present specification. does not

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, the same components are assigned the same reference numerals regardless of the reference numerals, and the overlapping description thereof will be omitted.

1 illustrates a feature map in a CNN model and a distribution of active values of pixels in the feature map.

Recently, as the size of a convolutional neural network (CNN) (hereinafter, 'CNN') has increased, the accuracy of the CNN has been improved. Modern CNN models require millions of computations and hundreds of megabytes of parameters. The CNN accelerator device stores parameters in an internal buffer through access to DRAM. Access to DRAM accounts for most of the energy consumption of CNN accelerator devices. The CNN accelerator device can minimize the number of DRAM accesses by compressing a feature map in order to reduce energy consumption. Additional power is consumed to compress the feature map, but the energy consumption for simple arithmetic operations or internal buffer access is much less than that of DRAM access.

Conventionally, for effective compression of a feature map, a focus has been placed on sparsity due to a rectified linear unit (ReLU) and quantization. To reduce computational complexity and solve the vanishing gradient problem, a ReLU activation function was used, and fixed-point data through quantization of the feature map was used. The activation function of Relu is a partial linear function that returns a zero value (0) when a negative value or zero value is input, and returns the corresponding positive value as it is when a positive value is input. Accordingly, since all negative values in the feature map become zero values, a feature map with many zero values, that is, a feature map with high sparsity, is derived. Quantization refers to converting a value expressed as a 32-bit floating point to an integer value using 32 bits or fewer than 32 bits. Through quantization, values close to zero can be quantized to zero, and a feature map with high sparseness can be derived. Conventionally, the feature map is compressed depending on only the feature of the feature map having high sparseness through relu or quantization.

Referring to FIG. 1 , a VGG-16 network 110 that is an exemplary model of one of the CNN models will be described. The feature maps 111 and 112 represent feature maps obtained based on the twelfth layer included in the VGG-16 network 110 . In the feature maps 111 and 112 , pixels having an active value of a zero value are displayed in white, and pixels having an active value of a non-zero value are displayed in gray or black. In the feature maps 111 and 112 , pixels having non-zero active values are clustered with each other. This indicates that the CNN activity values have spatial correlation. Accordingly, when the active value of a specific pixel is not a zero value, there is a high probability that the active values of a nearby pixel are also not zero. In general, pixels with active values of non-zero values in the feature maps 111 and 112 (eg, pixels marked in black) appear in small amounts, and in the feature maps 111 and 112 Pixels with an active value of zero (eg, pixels marked white) appear in large quantities.

The activation value distribution 120 represents the distribution of the activation values according to the maximum bit-width in the feature map of the VGG-16 network 110 . The maximum bit width may indicate the minimum number of bits required to express an active value. For example, an active value of 7 can be expressed as 00000111 ₍₂₎ in binary, with a maximum bit width of 3.

An active value of integer data of a pixel in the feature map may be classified into one of a zero value 121 , a non-outlier 122 , or an outlier 123 according to a maximum bit width, respectively. Most active values of non-zero values in the feature map may be expressed using N/2 bits in an N-bit fixed-point system. The non-outlier 122 may represent an active value expressible using N/2 bits in an N-bit fixed-point system. The outlier 123 may represent an activation value that cannot be expressed using N/2 bits in an N-bit fixed-point system. Here, N may represent a natural number of 1 or more. The zero value 121 has a maximum bit width of '0'. The non-outlier 122 has a maximum bit width of N/2 or less. The outlier 123 has a maximum bit width exceeding N/2. Non-outliers and outliers may be referred to as non-zero values.

Active values classified as outliers in the feature map occupy a very small part, but have a significant impact on the accuracy of CNNs. Since the outlier has to be expressed using a relatively large number of bits compared to the non-outlier, there is a limitation in quantization, which inevitably leads to a large data size. Conversely, a non-outlier can be expressed using only N/2 bits in an N-bit fixed-point system. Since most active values of pixels in the CNN feature map are non-outliers or zero values, representing data using fixed N bits is a waste of memory in terms of usage. However, there is a problem in that the bit width of the data representation cannot be reduced due to the outlier having to maintain a relatively long bit width. On the other hand, the compressor for compressing the feature map according to an embodiment may improve the compression ratio of the feature map by using spatial correlation between active values of the feature map and reducing the data representation bit width of the active values.

2 is a graph of spatial correlation between activation values within a specific distance within a feature map.

Using Moran's I, the spatial correlation between activity values within a feature map can be quantitatively evaluated. An example of an 8-bit quantized VGG-16 network trained based on ImageNet 2012 training data will be described. By performing inference in the VGG-16 network, a feature map may be obtained for each layer of the VGG-16 network. The compression ratio may be constantly evaluated depending on whether the active value is a zero value, but the spatial correlation may be evaluated differently depending on the size of the active value. In order to eliminate a difference in which spatial correlation is evaluated differently depending on the size of the active value, all non-zero values among the active values of the feature map may be changed to data of '1' and spatial correlation may be evaluated. The distance between the active values represents a distance between pixels corresponding to each of the corresponding active values in the feature map. Exemplarily, the distance between a pixel having a position of (x ₁ , y ₁ ) and a pixel having a position of (x ₂ , y ₂ ) in the feature map is |x ₂ -x ₁ |+|y ₂ -y ₁ It can be expressed as |. In order to investigate the spatial correlation between activation values within a specific distance d, the spatial coefficient w _ij may be defined as in Equation 1 below. Here, i and j represent indexes of active values in the feature map.

Moran's I may be calculated to evaluate the spatial auto correlation of the feature map obtained by converting the non-zero value into a value of '1'. Moran's I can be defined as in Equation 2 below.

및

는 각각 활성값 및 활성값의 평균을 나타낸다. Moran의 I는 -1 내지 1 사이의 값을 가진다. Moran의 I 값이 '-1'인 경우, 논-제로 값의 활성값들이 완벽하게 분산되어 있음을 나타내고, Moran의 I 값이 '1'인 경우, 논-제로 값의 활성값들이 완벽하게 군집되어 있음을 나타낸다. 일반적으로 특정 데이터에 대한 Moran의 I가

보다 크다면, 특정 데이터가 양의 공간적 상호 관계성을 가지는 것으로 판단된다.In Equation 2, L represents the total number of active values in the feature map.

and

represents the active value and the average of the active values, respectively. Moran's I has a value between -1 and 1. When Moran's I value is '-1', it indicates that the active values of non-zero values are perfectly distributed, and when Moran's I value is '1', the active values of non-zero values perfectly cluster. indicates that it has been In general, Moran's I for a particular data is

If larger, it is determined that the specific data has a positive spatial correlation.

보다 큰 것으로 나타나기 때문에, 특징 맵의 활성값들은 양의 공간적 상호 관계성을 갖는 것으로 판단된다. 이하에서는, 특징 맵의 높은 공간적 상호 관계성을 이용하기 위하여, 타일(tile)을 기반으로 특징 맵을 컴프레싱하는 방법에 관하여 설명한다.2 shows the spatial correlation (Moran's I) between activation values within a specific distance calculated for different convolutional layers. The graph 211 is a graph regarding the spatial correlation between active values with a distance of '1', the graph 212 is a graph about the spatial correlation between active values with a distance of '2', and the graph ( 213 is a graph relating to spatial correlation between active values having a distance of '3', and graph 214 is a graph relating to spatial correlation between active values having a distance of '4'. Referring to FIG. 2 , it appears that the spatial correlation between activation values within a close distance is higher. The average Moran's I between active values with a distance of '1' appears to be 0.54, and the average Moran's I between active values appears to decrease to 0.35, 0.29, and 0.26 as the distance increases. Spatial interrelationship is always at all distances

Since it appears larger, it is judged that the activation values of the feature map have positive spatial correlation. Hereinafter, a method of compressing a feature map based on a tile in order to utilize the high spatial correlation of the feature map will be described.

3 is a flowchart illustrating an operation of a neural network accelerator device according to an embodiment.

In operation 310, the neural network accelerator device (hereinafter, 'accelerator device') according to an embodiment may acquire a feature map based on one or more layers included in the CNN model.

In operation 320, the accelerator device according to an embodiment may divide the acquired feature map into unit tiles having a predetermined size, and sequentially input the divided unit tiles into a register of the compressor. . The accelerator device may divide the feature map into a plurality of unit tiles in order to utilize the high spatial interrelationship of the feature map. For example, the feature map may be divided into m×n unit tiles, and an active value in the unit tiles may be compressed. The width and height of the unit tile may be predetermined. An m×n tile may indicate a tile in which m pixels are arranged along one axis and n pixels are arranged in another axis perpendicular to one axis.

In operation 330 , the accelerator device according to an embodiment performs first data (also referred to as a run) indicating the number of zero tiles input to the register before the non-zero tiles are input to the register, and non-zero tiles. second data (also referred to as a mask) indicating a pixel having a zero value and a pixel having a non-zero value in the tile, and an activation value of the pixel having a non-zero value in the non-zero tile ), the third data may be generated.

The accelerator device according to an embodiment may classify the unit tile input to the register of the compressor into a zero tile or a non-zero tile. The zero tile may indicate a tile in which all active values of pixels within the tile are zero values. The non-zero tile may indicate a tile in which at least one of active values of pixels in the tile is a non-zero value. The accelerator device may count the number of zero tiles input to the register before the non-zero tiles are input to the register, and generate first data indicating a counting result. The first data may indicate the number of consecutive zero tiles. For example, when two consecutive zero tiles are input to the register, the first data is 10 ₍₂₎ . A bit width of the first data may be predetermined.

)에 의하여 결정될 수 있다.The accelerator device according to an embodiment may generate second data indicating a pixel having a zero value and a pixel having a non-zero value within a non-zero tile input to a register. The bit width of the second data is the size of the unit tile (

) can be determined by

The accelerator device according to an embodiment may generate third data based on an active value of a pixel having a non-zero value in a non-zero tile. The accelerator device may generate third data by excluding active values of pixels having a zero value in the non-zero tile and extracting only active values of pixels having a non-zero value.

In operation 340, the accelerator device according to an embodiment may store the generated first data, second data, and third data in a buffer. The compressor can output the stored data to DRAM.

4 illustrates a method of compressing a feature map by an accelerator device according to an embodiment.

Hereinafter, a first compression method 440 according to an embodiment among various methods for compressing the feature map will be described. A compressor of the accelerator device according to an embodiment may divide the feature map 410 into unit tiles, and classify active values of pixels in the divided unit tiles into zero or non-zero values. can do. The compressor according to an exemplary embodiment includes first data (eg, 10 _{( 2)} ) may be generated as the first data 441 . The bit width of the first data may be predetermined.

The compressor generates second data 442 indicating a pixel 431 having an active value of zero and pixels 432 , 433 , and 434 having an active value of a non-zero value in the non-zero tile 423 . can create For example, the compressor may determine that the pixel 431 having an active value of zero indicates '0 ₍₂₎ ', and the pixels 432, 433, 434 having an active value of non-zero value are It can be determined by indicating '1'. The bit width of the second data 442 may be determined according to the size of the unit tile. For example, when the size of the unit tile is determined to be 2×2, the compressor may determine the second data 442 to have a 4-bit width. In this case, the compressor may generate 0111 ₍₂₎ as the second data.

The compressor according to an embodiment may generate the third data 443 based on the pixels 432 , 433 , and 434 having a non-zero value in the non-zero tile 423 . According to an embodiment, the compressor may generate an active value of the pixels 432 , 433 , and 434 having a non-zero value as data having a fixed first bit width. For example, in an N-bit fixed-point system, the first bit width may represent an N bit width.

Furthermore, the compressor according to an embodiment sets the threshold value to the first data when the count of the first data indicating the number of zero tiles input to the register reaches the threshold value before the non-zero tiles are input to the register. 441 , and a zero value may be generated as the second data 442 . Here, the threshold value may represent a maximum value that can be expressed according to a predetermined bit width of the first data. For example, when the predetermined bit width for the first data is 2 bits, the threshold value is determined to be 3 ( 11 _{( 2 )} ). For example, when the number of consecutive zero tiles input to the register is three, the compressor may generate the first data as 11 ₍₂₎ and generate the second data as 0000 ₍₂₎ . The compressor repeats generation and storage of data until all active values of unit tiles of the feature map are compressed, and finally generates and stores an end of packet 444 . End packet 444 may be zero data.

Hereinafter, a second compression method 450 according to an embodiment among various methods for compressing the feature map will be described. The compressor of the accelerator device may further improve the compression ratio of the feature map by reducing the bit width of the data representation of the active values. The compressor may compress the feature map by classifying the active value of the feature map as one of a zero value, a non-outlier, and an outlier.

First, in the compressor according to the embodiment, first data indicating the number of zero tiles 421 and 422 input to the register before the non-zero tile 423 is input to the register (eg, 10(2)) may be generated as the first data 451 .

The compressor classifies an active value of a pixel in the non-zero tile 423 as a zero value, a non-outlier, or an outlier to generate second data 452 indicating each pixel. As described above, a non-outlier can represent an active value that can be expressed using N/2 bits in an N-bit fixed-point system, and an outlier can represent an active value that cannot be expressed using N/2 bits. have.

The compressor may classify pixels in the non-zero tile 423 according to an active value. The compressor may map a first value to a pixel 431 having an active value of zero in the non-zero tile 423 , and a second value to the pixel 432 , 433 having an active value of a non-outlier. A value may be mapped, and a third value may be mapped to the pixel 434 having the active value of the outlier. For example, the compressor may determine that a pixel 431 having an active value of zero in the non-zero tile 423 indicates '00 ₍₂₎ ', and having an active value of a non-outlier It may be determined that the pixels 432 and 433 indicate '01 ₍₂₎ ', and it may be determined that the pixel 434 having the active value of the outlier indicates '10 ₍₂₎ '. The compressor may generate the second data 452 by connecting data indicated by each pixel. In the second data 452 , an additional bit is required to classify an active value as a zero value, a non-outlier, or an outlier, compared to a case for classifying the active value into a zero value or a non-zero value. . However, since most active values of non-zero values are non-outliers, it is possible to generate the third data 453 using a smaller bit width, which is more effective than the first compression method 440 . compression ratio can be expressed.

The compressor according to an embodiment may generate the third data 453 based on the pixels 432 , 433 , and 434 having a non-zero value in the non-zero tile 423 . The compressor according to an embodiment represents an active value of the pixel 434 having an outlier as data of a first bit width, and sets the active value of the pixels 432 and 433 having a non-outlier as smaller than the first bit width. The third data may be generated by representing it as data having a second bit width. For example, in an N-bit fixed-point system, a first bit width may represent an N bit width, and a second bit width may represent an N/2 bit width. According to an embodiment, when the compressor generates the third data 453 , the active value of the pixel having the non-outlier may be represented as data having a width of N/2 bits. Since the compressor can represent the active value of a pixel having a non-outlier by using a bit width equal to half of the existing fixed N-bit width, compression efficiency can be increased.

The compressor repeats generation and storage of data until all active values of unit tiles of the feature map are compressed, and finally generates and stores an end of packet 454 . End packet 454 may be zero data.

5 illustrates a configuration of an accelerator device according to an exemplary embodiment.

The accelerator device 510 according to an embodiment includes a processing element array (PE array) 511 , an on-chip global buffer 512 , a controller 513 , and a compressor ( 514 ), and a decompressor 515 .

The processing element array 511 may perform a convolutional neural network operation. Appropriate data from the off-chip memory 520 (eg, DRAM) is decompressed by the decompressor 515 to perform the convolutional neural network operation of the processing element array 511 , and the decompressed data is transferred to the on-chip global It may be loaded into the buffer 512 . Data calculated through the convolutional neural network operation in the processing element array 511 is transmitted to and stored in the on-chip global buffer 512 , and data compressed through the compressor 514 is transmitted to the off-chip memory 520 . can be saved. Hereinafter, the structure of the compressor will be described in more detail in FIG. 6 , and the structure of the decompressor will be described in more detail in FIG. 7 .

6 illustrates a structure of a compressor according to an embodiment.

The compressor according to an embodiment may compress the feature map. The compressor 600 according to an embodiment may include a register 601 , a comparator 602 , a buffer 603 , and a controller 613 .

The accelerator device may divide the feature map into unit tiles having a predetermined size with respect to the feature map obtained from the input image based on one or more layers included in the convolutional neural network model. The register 601 of the compressor according to an embodiment may sequentially receive the divided unit tiles of the feature map. Active values of pixels included in the unit tile may be input to the register 601 . The comparator 602 may compare data input and stored in a register. Hereinafter, the second compression method 450 according to an embodiment will be described along with the structure of the compressor 600 .

In a first cycle, active values Din ₀ , Din _{1 , ...,} Din _k-1 of the unit tile may be input to and stored in the parallel register 601 of the compressor. According to an embodiment, an active value of a pixel may be divided into high-order bit data 641 and low-order bit data 642 and input to the parallel register 601 of the compressor. The controller of the compressor may divide the active value of the pixel into high-order bit data 641 and low-order bit data 642 and input it to the register 601 . The high-order bit data 641 may indicate data corresponding to a bit position from an N/2 bit position to a most significant bit (MSB) position. High-order bit data 641 may represent data corresponding to high-order N/2 bits when an active value is expressed by N bits in an N-bit fixed-point system. The low-order bit data 642 may indicate data corresponding to bit positions from a least significant bit (LSB) to an N/2 bit position. The low-order bit data 642 may represent data corresponding to the low-order N/2 bits when an active value is expressed by N bits.

The comparator 602 may determine whether the higher bit data 641 and the lower bit data 642 of the active value corresponding to one pixel are zero values, respectively. The controller 613 may classify the active value as one of an outlier, a non-outlier, or a zero value based on the comparison result. Based on the comparison result, the controller of the compressor may classify the active value as an outlier when both the upper bit data 641 and the lower bit data 642 of the active value corresponding to one pixel are non-zero values. . Similarly, when the high-order bit data 641 is a zero value and the low-order bit data 642 is a non-zero value, the controller of the compressor classifies the active value as a non-outlier, and the high-bit data 641 and the low-order bit data When all (642) is a zero value, the active value may be classified as a zero value.

In the next cycle, the compressor 600 according to an embodiment may generate the data 631 based on the active value of the pixel having the non-zero value among the input active values of the unit pixel. The controller may sequentially extract non-zero values through a shift-and-append logic that removes a zero value among active values of a pixel. The compressor uses a mux 604 to represent an active value classified as an outlier as first bit-width data, and an active value classified as a non-outlier is data of a second bit width that is smaller than the first bit width By representing , the third data 630 may be generated.

In the last cycle, the compressor may store the first data 610 , the second data 620 , and the third data 630 in the buffer 603 . The controller of the compressor according to an embodiment corresponds to the first value the active value classified as a zero value according to the comparison result of the comparator, and the active value classified as non-outlier to the second value, and to the outlier The second data 620 may be generated by matching the classified activity value to the third value. The controller maps the first value to the pixel with the active value classified as a zero value according to the comparison result of the comparator, maps the second value to the pixel with the active value classified as a non-outlier, and maps the active value classified as an outlier. The second data 620 may be generated by mapping the third value to the pixel of the value. The first value, the second value, and the third value may be different data. For example, the first value may be '00 ₍₂₎ ', the second value may be '01 ₍₂₎ ', and the third value may be '10 ₍₂₎ '. The second data 620 may be generated by sequentially connecting values indicated by each pixel. Here, the number of bits of the second data may be twice the number of pixels included in the unit tile.

The controller of the compressor according to an embodiment may classify the input unit tile into a zero tile or a non-zero tile. The controller of the compressor may determine that the input unit tile is a zero tile when all of the input active values of the unit tile are zero values. More specifically, the controller may determine whether the input unit tile is a zero tile using the second data 620 indicating each pixel within the unit pixel. The compressor 600 may determine whether the sum 611 of values indicated by each pixel within the input unit tile is zero. When the sum 611 of the values indicated by each pixel is a zero value, the compressor 600 may determine that the input unit tile is a zero tile. When the sum of values indicated by each pixel is a non-zero value, the compressor may determine that the input unit tile is a non-zero tile. When it is determined that the input unit pixel is a zero tile, the compressor according to an embodiment calculates a count 612 of first data indicating the number of zero tiles input to the register before the non-zero tile is input to the register. It can be increased by '1'. In other words, when the sum 611 of the values indicated by each pixel of the input unit tile is zero, the compressor may increase the count 612 of the first data by '1'. When the sum 611 of values indicated by each pixel of the input unit tile is non-zero, the controller 613 may determine that the input unit tile is a non-zero tile. When the non-zero tile is input to the register, the controller of the compressor generates the first data 610 according to the value stored in the count 612 of the first data, and stores the generated first data 610 in the buffer 603 ) can be stored in Also, when the non-zero tile is input to the register, the compressor may store the generated second data 620 in the buffer 603 and store the third data 630 in the buffer 603 .

The delay time for compressing the unit tile varies from 3 to 4+k cycles. Here, k may represent the number of pixels included in the unit tile. The minimum period appears when a zero tile is input to the register, and the maximum period appears when the active values of all pixels of the unit tile are non-zero values. When n compressors are used, the time to compress the feature map is reduced by n times. Here, n may represent a natural number.

7 illustrates a structure of a decompressor for decompressing a feature map according to an embodiment.

The decompressor 700 (hereinafter, 'decompressor') for decompressing the feature map according to an embodiment may decompress the data obtained by compressing the feature map obtained based on the layer of the CNN model. Data obtained by compressing the feature map may be input to the register of the decompressor.

In the first cycle, the first data 710 and the second data 720 may be simultaneously input to and stored in the register of the decompressor 700 . In the next cycle, the third data 730 may be input to and stored in the register of the decompressor 700 . The decompressor 700 may store the third data 730 indicating the active value of the non-zero value in the register in the order in which it is input. In the next cycle, the controller 713 of the decompressor may generate and transmit the number of zero tiles corresponding to the first data 710 to the buffer. In the last cycle, the controller of the decompressor may generate a non-zero tile by using the second data 720 and the third data 730 and transmit it to the buffer. The controller 713 of the decompressor may distinguish a pixel having a zero value and a pixel having a non-zero value in the non-zero tile using the second data 720 . The controller 713 of the decompressor may generate a non-zero tile by sequentially matching the active value of the pixel having a zero value with the zero value, and sequentially matching the active value of the pixel having a non-zero value with the third data. have. The decompressor repeats the above sequence until the end packet is transmitted.

The delay time for decompressing the feature map compression data varies from 3 to 3+1 cycles. Here, l may represent the maximum magnitude of the first data. The minimum period may appear when the end packet is decompressed, and the maximum period may appear when the first data has a maximum size. When n decompressors are used, the time for decompressing the feature map compression data is reduced by n times. Here, n may represent a natural number.

Table 1 below shows hardware sizes and power consumption for a compressor and a decompressor according to an embodiment.

Device

]size[

] Power consumption [mW] Compressor according to fig. 11,124 0.488 Decompressor according to Fig. 7 4,060 0.24 MAC (multiply-accumulate) 1,283 0.248 256 MACs 328,448 63.4888

In Table 1, along with the size and power consumption of hardware for the compressor and decompressor according to an embodiment, the size and power consumption for the MAC (multiply-accumulate) unit in the CNN accelerator device, the number used in the general CNN accelerator device ( For example, it represents the size and power consumption of the entire MAC unit as much as 256). The MAC unit consists of an 8-bit multiplier and a 28-bit accumulator. The size and power consumption of the compressor and the decompressor according to an embodiment are similar to that of one MAC unit. Since the MAC unit occupies a very small part in the CNN accelerator device, the size and power consumption of the compressor or decompressor according to an embodiment are very small. The size and power consumption of the compressor or decompressor according to an embodiment are 4.62% and power consumption of only about 1.15% compared to a general CNN accelerator device.

8 shows a graph of the reciprocal of normalized DRAM access according to the size of a unit tile.

Hereinafter, a result of compressing or decompressing a feature map using a compressor or a decompressor by the accelerator device according to an exemplary embodiment will be described. Feature maps can be inferred using three CNN networks: VGG16, ExtractionNet, and ResNet-18. The three CNN feature maps can be quantized to k bits using Equation 3 below.

A compression ratio for a method of compressing the feature map is defined as in Equation 4 below.

The compression ratio is expressed as a positive rational number, but the compression rate of uncompressed data is expressed as '1'.

)에 따른 정규화된 DRAM 액세스의 역수에 대한 그래프를 나타낸다. 그래프(820)은 ExtractionNet 네트워크에서 단위 타일의 크기에 따른 정규화된 DRAM 액세스의 역수에 대한 그래프를 나타낸다. 그래프(830)은 ResNet-18 네트워크에서 단위 타일의 크기에 따른 정규화된 DRAM 액세스의 역수에 대한 그래프를 나타낸다.A compression effect of a method of compressing a feature map or a method of decompressing a feature map according to an embodiment may be evaluated using the reciprocal of normalized DRAM accesses. 8 shows a graph of the reciprocal of normalized DRAM access according to the size of a unit tile in three CNN networks. The graph 810 shows the size of the unit tile in the VGG-16 network (

) shows a graph for the inverse of normalized DRAM access according to The graph 820 shows a graph of the reciprocal of normalized DRAM access according to the size of a unit tile in the ExtractionNet network. The graph 830 shows a graph of the reciprocal of normalized DRAM access according to the size of a unit tile in the ResNet-18 network.

Each DRAM access represents the average number of accesses of a feature map quantized to various bits (eg, a feature map quantized to 4 to 8 bits), and the average number of accesses for each is normalized to the data size before compression. In the VGG-16 network, the number of DRAM accesses is smallest when m and n are 2. When m or n has a value greater than 2 in the VGG-16 network, the number of DRAM accesses appears to increase. When a feature map is segmented using a large unit tile, many zeros can be compressed with zero tiles at once. The number of is rapidly decreased, resulting in poor compression ratio. The cause of this tendency is that, referring to FIG. 2, the average spatial correlation (Moran's I) in the VGG-16 network is greatly reduced when the distance between active values is '3' or more. Accordingly, when the distance between active values, that is, the horizontal length or the vertical length of the unit tile is '3' or more, the number of zero tiles is reduced. Furthermore, since a unit tile having a larger size requires a large amount of second data having a bit width, the size of the unit tile should be determined in consideration of spatial interrelationship. As a result, the number of DRAM accesses is 0.36 in the VGG-16 network, which is the minimum at m=2 and n=2. In addition, referring to the graph 820 and the graph 830, the number of DRAM accesses in the ExtractionNet network and the ResNet-18 network is 0.37 and 0.34, respectively, respectively, which is the minimum at m=2 and n=2.

9 is a graph showing a compression ratio and sparsity of a feature map according to various compression methods.

The graph 910 represents a graph regarding the compression ratio and sparsity of the feature map for each layer according to the compression method in the VGG-16 network. The graph 920 represents a graph regarding the compressibility and sparsity of the feature map for each layer according to the compression method in the ExtractionNet network. The graph 930 shows a graph regarding the compression ratio and sparsity of the feature map for each layer according to the compression method in the ResNet-18 network.

Graph 911 , graph 912 , graph 913 , graph 914 , and graph 915 are RLC8, RLC4, ZVC, first compression method 440, and second compression method in the VGG-16 network, respectively. It represents the compression ratio of the feature map for the layer according to the pressing method 450 . Graph 916 represents a graph regarding the sparsity of feature maps for layers in the VGG-16 network.

Run-length compression (RLC) is a method of lowering DRAM access by expressing consecutive zero tiles in a feature map as one run data. RLC stores non-zero data after run data. RLC4 and RLC8 indicate a compression method using 4 bits of run data and 8 bits of run data, respectively. ZVC (Zero-Value Compression) is a method of storing non-zero data in a feature map and storing mask data indicating a position value of the corresponding data.

Referring to FIG. 9 , the compression ratio shows a significant correlation with the sparsity of the feature map. In the back layer, the sparseness of the feature map increases. Therefore, it appears that the deeper the layer, the higher the compression rate of all algorithms. The second compression method 450 according to an embodiment shows the highest average compression ratio of 2.77, and the average compression ratios of RLC8, RLC4, and ZVC are 1.60, 1.86, and 2.21, respectively.

Due to the low sparseness of the feature map in a specific layer, the compression ratios of RLC8 and RLC4 are less than 1.0. However, the compression ratio according to the ZVC, the first compression method 440 and the second compression method 450 always appears as 1.0 or more. From the 8th layer, the difference between the compression ratio of the first compression method or the second compression method and the compression ratio of the ZVC gradually increases, which occurs when the sparsity of the feature map increases. This is because the pressing method can effectively compress a zero value using a non-zero tile. Although it is shown that ZVC has a higher compression ratio than that of the first compression method 440 in some layers (eg, the 2nd, 3rd, 4th, and 7th layers), ZVC and the first Compression The difference in compression ratio between the pressing methods 440 is very small, 0.04 or less.

Graph 921, graph 922, graph 923, graph 924, and graph 95 are RLC8, RLC4, ZVC, first compression method 440, and second compression method, respectively, in the ExtractionNet network. It represents the compression ratio of the feature map for the layer according to (450). Graph 926 represents a graph regarding the sparsity of feature maps for layers in the ExtractionNet network. In the ExtractionNet network, the compression ratio according to the second compression method 450 represents the best compression ratio. The average compression ratios of RLC8, RLC4, ZVC and the second compression method 450 for ExtractionNet are 1.73, 1.97, 2.32 and 2.84, respectively.

Graph 931 , graph 932 , graph 933 , graph 934 , and graph 935 are RLC8, RLC4, ZVC, first compression method 440, second compression method, respectively, in the ResNet-18 network. It represents the compression ratio of the feature map for the layer according to the pressing method 450 . Graph 926 represents a graph regarding the sparsity of feature maps for layers in the ResNet-18 network. The average compression rates of RLC8, RLC4, ZVC, the first compression method 440 and the second compression method 450 are 2.21, 2.36, 2.66, 3.30, and 3.52, respectively. The reason ResNet-18's rarity fluctuates is the effect of the shortcut-layer.

Figure 10 shows the values of normalized DRAM accesses of the compressed feature map with respect to quantized activation values.

Graph 1010, graph 1020, and graph 1030 are the normalized DRAM access values of the feature map compressed with respect to active values quantized from 4 bits to 8 bits in the VGG-16 network, the Extraction network, and the ResNet network, respectively. indicates

Graph 1011, Graph 1012, Graph 1013, Graph 1014, Graph 1015 are RLC8, RLC4, ZVC, First Compression Method 440, Second Compression, respectively, in the VGG-16 network. A graph is shown for values of normalized DRAM accesses of a compressed feature map according to method 450 .

Normalized DRAM access values of the compressed feature map are measured for quantized activation values from 4 bits to 8 bits. Except for the 4-bit quantized feature map, the second compression method 450 exhibits the lowest normalized DRAM access value. The first compression method 440 represents a decrease in the highest DRAM access value in the feature map quantized to 4 bits. In the 8-bit quantized feature map, the second compression method 450 shows a reduction in DRAM access value of 50%, and ZVC, RLC4, and RLC8 show a reduction in DRAM access value of 39%, 20%, and 10%, respectively. indicates. When the feature map is quantized to 8 bits or less, the DRAM access value decreases regardless of the compression method. This is because the compression ratio increases as the sparseness of the feature map increases. This reduction in DRAM access values is smallest in ZVC because it uses a fixed size of mask data. The first compression method 440 and the second compression method 450 may effectively compress the quantized feature map. In particular, in the feature map quantized to 4 bits, the first compression method 440 and the second compression method 450 may reduce DRAM access values to 82% and 80%. Also, the first compression method 440 shows a smaller DRAM access value in the feature map quantized to 4 bits, which means that the feature map is sufficiently sparse and the second compression method 450 converts the active value into an outlier, non -Outlier, this is because the second data (mask) with a larger bit width is used to express it as a zero value.

Graph 1011, graph 1012, graph 1013, graph 1014, and graph 1015 are RLC8, RLC4, ZVC, first compression method 440, second compression method ( 450) shows a graph of the values of normalized DRAM accesses of the compressed feature map.

Graph 1011, Graph 1012, Graph 1013, Graph 1014, Graph 1015 are RLC8, RLC4, ZVC, First Compression Method 440, Second Compression, respectively, in the ResNet-18 network. A graph is shown for values of normalized DRAM accesses of a compressed feature map according to method 450 .

In ExtractionNet and ResNet-18, DRAM access values show a similar trend to that of VGG-16. In the 8-bit quantized feature map, the second compression method 450 reduces DRAM access values by 54% and 57%, respectively, and the first compression method reduces DRAM access values by 48% and 50%, respectively. In the feature map quantized to 4 bits, the second compression method 450 reduces the DRAM access value by 76% and 78%, respectively, and the first compression method 440 reduces the DRAM access value by 78% and 80%, respectively .

11 shows a graph of energy consumed to store and load a feature map.

Graph 1111, graph 1121, and graph 1131 relate to the energy consumed to store and load the feature map when compressing the feature map using RLC4 in VGG-16, ExtractionNet, and ResNet-18 networks. show the graph. Graph 1112, graph 1122, and graph 1132 relate to the energy consumed to store and load the feature map when compressing the feature map using ZVC in the VGG-16, ExtractionNet, and ResNet-18 networks. show the graph. The graph 1113, the graph 1123, and the graph 1133 store the feature map and Shows a graph of the energy consumed to load.

Energy consumption is normalized to the CNN accelerator device without a compressor. As mentioned earlier, DRAM access is responsible for most of the energy consumption. Accordingly, in the compression method according to an embodiment, energy consumption for storing and loading the feature map is reduced compared to RLC4 and ZVC, even though the energy consumption of hardware is higher than that of RLC4 and ZVC. The compression method according to an embodiment reduces energy consumption for DRAM access in VGG-16, ExtractionNet and ResNet-18 networks to 0.35, 0.38 and 0.40, respectively. As a result, the compression method according to an embodiment may reduce energy consumption by an average of 37.7%.

The embodiments described above may be implemented by a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the apparatus, methods and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA) array), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, may be implemented using a general purpose computer or special purpose computer. The processing device may execute an operating system (OS) and a software application running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that may include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

The software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in a computer-readable recording medium.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination, and the program instructions recorded on the medium are specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. may be Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

The hardware devices described above may be configured to operate as one or a plurality of software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with reference to the limited drawings, a person skilled in the art may apply various technical modifications and variations based thereon. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (20)

A method of compressing a feature map performed by a compressor, the method comprising:
obtaining a feature map from an input image based on one or more layers included in a convolutional neural network (CNN) model;
dividing the acquired feature map into unit tiles of a predetermined size, and sequentially inputting the divided unit tiles into a register;
First data indicating the number of zero tiles input to the register before the non-zero tiles are input to the register, indicating a pixel having a zero value and a pixel having a non-zero value within the non-zero tile generating third data based on the second data and an activation value of a pixel having a non-zero value in the non-zero tile; and
storing the first data, the second data, and the third data in the buffer;
A feature map compression method comprising:
According to claim 1,
The step of generating the data includes:
When all pixels within the unit tile input to the register have zero values, the count of first data indicating the number of zero tiles input to the register is increased before the non-zero tiles are input to the register step
A feature map compression method comprising:
According to claim 1,
When the count of first data indicating the number of zero tiles input to the register reaches a threshold before the non-zero tiles are input to the register, a threshold value is generated as the first data, and a zero value is generated. generating as the second data
A feature map compression method further comprising a.
According to claim 1,
The step of generating the data includes:
generating the third data by representing an active value of a pixel having a non-zero value in the non-zero tile as data having a first bit width
A feature map compression method comprising:
According to claim 1,
The step of generating the data includes:
classifying active values of pixels in the non-zero tile into one of an outlier, a non-outlier, and a zero value; and
generating the third data by representing an active value classified as an outlier as data having a first bit width, and representing an active value classified as a non-outlier as data having a second bit width smaller than the first bit width step
A feature map compression method comprising:
6. The method of claim 5,
The step of inputting the divided unit tiles into a register comprises:
dividing the active value of each pixel included in the unit tile into high-order bit data and low-order bit data and inputting them into the register
including,
The step of generating the data includes:
classifying the active value into one of an outlier, a non-outlier, and a zero value by determining whether the upper bit data and the lower bit data corresponding to the active value of the pixel are zero values
A feature map compression method comprising:
7. The method of claim 6,
The step of classifying the active value is,
classifying the active value as an outlier when both the upper bit data and the lower bit data are non-zero values;
classifying the active value as a non-outlier when the upper bit data is a zero value and the lower bit data is a non-zero value; and
Classifying the active value as a zero value when both the upper bit data and the lower bit data are zero values
A feature map compression method comprising:
6. The method of claim 5,
The step of generating the data includes:
In the non-zero tile, a first value is mapped to a pixel having a zero value, a second value is mapped to a pixel having a non-outlier, and a third value is mapped to a pixel having an outlier so that the second value is mapped to the second value. Steps to generate data
including,
wherein the first value, the second value, and the third value are different data from each other,
Feature map compression method.
A computer-readable storage medium storing one or more computer programs comprising instructions for performing the method of any one of claims 1 to 8.
A compressor for compressing a feature map, comprising:
With respect to a feature map obtained from an input image based on one or more layers included in the convolutional neural network model, the feature map is divided into unit tiles of a predetermined size, and the divided unit tiles are sequentially register input to;
a comparator for comparing data stored in the register;
First data indicating the number of zero tiles input to the register before a non-zero tile is input to the register based on the result of the comparison, a pixel having a zero value in the non-zero tile and a non-zero tile a controller for generating third data based on second data indicating a pixel having a value, and an active value of a pixel having a non-zero value within the non-zero tile; and
A buffer for storing the first data, the second data, and the third data, and outputting the stored data to a dynamic random access memory (DRAM)
Compressor containing
11. The method of claim 10,
The controller is
When all pixels of the unit tile input to the register have a zero value, increasing the count of first data indicating the number of zero tiles input to the register before the non-zero tile is input to the register;
Compressor.
11. The method of claim 10,
The controller is
When the count of first data indicating the number of zero tiles input to the register reaches a threshold before the non-zero tiles are input to the register, a threshold value is generated as the first data, and a zero value is generated. to generate as the second data
Compressor.
11. The method of claim 10,
The controller is
generating the third data by representing an active value of a pixel having a non-zero value in the non-zero tile as data having a first bit width
Compressor.
11. The method of claim 10,
The controller is
classifying an active value of pixels in the non-zero tile into one of an outlier, a non-outlier, and a zero value;
generating the third data by representing an active value classified as an outlier as data having a first bit width, and representing an active value classified as a non-outlier as data having a second bit width smaller than the first bit width
Compressor.
15. The method of claim 14,
The register is
receiving an input in which an active value of each pixel included in a unit tile is divided into high-order bit data and low-order bit data;
The comparator is
Determining whether high-order bit data and low-order bit data corresponding to the active value of the pixel are zero values;
The controller is
classifying an active value into one of an outlier, a non-outlier, and a zero value based on the determination result;
Compressor.
16. The method of claim 15,
The controller is
Based on the comparison result, when both high-order bit data and low-order bit data are non-zero values, an active value is classified as an outlier, and when the high-order bit data is a zero value, and the low-order bit data is a non-zero value, an active value is classified as non-outlier, and when both high-order bit data and low-order bit data are zero values, the active value is classified as a zero value.
Compressor.
15. The method of claim 14,
The controller is
In the non-zero tile, a first value is mapped to a pixel having a zero value, a second value is mapped to a pixel having a non-outlier, and a third value is mapped to a pixel having an outlier so that the second value is mapped to the second value. create data,
The first value, the second value, and the third value are different data from each other,
Compressor.
In the neural network accelerator device,
a buffer for obtaining a feature map extracted from an input image based on one or more layers included in the convolutional neural network model using a processing element array (PE array) that performs a convolutional neural network operation;
a controller dividing the acquired feature map into unit tiles of a predetermined size and sequentially inputting the divided unit tiles into a compressor; and
First data indicating the number of zero tiles input to the register before the non-zero tiles are input to the register, the first data indicating a pixel having a zero value and a pixel having a non-zero value within the non-zero tile generate third data based on two data and an active value of a pixel having a non-zero value in the non-zero tile, and store the first data, the second data, and the third data; Compressor that outputs stored data to dynamic random access memory (DRAM)
A neural network accelerator device comprising a.
A method of decompressing a feature map performed by a decompressor, comprising:
inputting the compressed data of the feature map obtained based on the layer of the convolutional network model into a register; and
generating and transmitting the number of zero tiles corresponding to the first data to the buffer, and generating and transmitting non-zero tiles based on the second data and the third data to the buffer;
including,
The step of generating the non-zero tile and transmitting it to the buffer comprises:
A pixel having a zero value and a pixel having a non-zero value are distinguished from a pixel having a non-zero value in the non-zero tile by using the second data, and an active value of the pixel having a zero value corresponds to the zero value, and a non-zero value is used. generating the non-zero tile by sequentially matching an active value of a pixel having a value with the third data;
A feature map decompressing method comprising a.
A decompressor for decompressing a feature map, comprising:
a register to which compressed data of a feature map obtained based on a layer of a convolutional network model is input;
A controller that generates and transmits the number of zero tiles corresponding to the first data to the buffer, and generates non-zero tiles based on the second data and the third data and transmits them to the buffer
including,
The controller is
A pixel having a zero value and a pixel having a non-zero value are distinguished from a pixel having a non-zero value in the non-zero tile by using the second data, and an active value of the pixel having a zero value corresponds to the zero value, and a non-zero value is used. The active value of the pixel having a value is sequentially matched with the third data to generate the non-zero tile.
decompressor.

Images