KR102277644B1 - Conv-xp pruning apparatus of convolutional neural network suitable for an acceleration circuit - Google Patents

Abstract

A Conv-XP pruning device for a convolutional neural network suitable for an acceleration circuit is disclosed. An arithmetic circuit of a convolutional neural network, comprising: a mux (100) for selecting an X array or a + array from input data (10); a multiplier 200 that multiplies the output of the mux 100 and the weight of the neural network; and an adder 300 that adds the output of the multiplier 200 . Therefore, it is possible to make a high-speed circuit by reducing the number of multipliers 200 , and there is an advantage in that the area of the acceleration circuit in the operation circuit is reduced.

Description

Conv-XP Pruning Apparatus for Convolutional Neural Networks Suitable for Acceleration Circuits {CONV-XP PRUNING APPARATUS OF CONVOLUTIONAL NEURAL NETWORK SUITABLE FOR AN ACCELERATION CIRCUIT}

The present invention relates to a Conv-XP pruning device for a convolutional neural network suitable for an acceleration circuit, and more particularly, to a Conv-XP pruning device for a convolutional neural network suitable for an acceleration circuit that reduces the number of multipliers.

In a convolutional neural network, input data and weights are multiplied by a multiplier, and the multiplication result is added by an adder to output. The arithmetic circuit of a convolutional neural network consists of a multiplier that multiplies the input data and weight and an adder that adds the multiplication result. An example of such an arithmetic circuit is as follows.

1 is an exemplary diagram illustrating an operation circuit of a conventional convolutional neural network.

In the conventional operation circuit of a convolutional neural network, 9 multipliers multiply 3*3 input data and 3*3 weights, and then add them by an adder. Such an arithmetic circuit has nine multipliers, and the multiplier has a problem in that the area of the entire arithmetic circuit increases because of its high complexity.

2 is an exemplary diagram illustrating another embodiment of a conventional convolutional neural network.

An operation circuit of a conventional convolutional neural network has 5 muxes for multiplexing 3*3 input data, 5 multipliers for multiplying 5 mux outputs and 5 weights, and has one adder. Although this arithmetic circuit has five multipliers, there is a problem in that the mux also has five, increasing the complexity of the arithmetic circuit.

Publication No. 10-2018-0123846, Reconstructed Computational Accelerator of Logical 3D Structure for Convolutional Neural Network

An object of the present invention for solving the above problems is to provide a Conv-XP pruning device of a convolutional neural network suitable for an acceleration circuit that reduces the number of multipliers.

The present invention for achieving the above object, in an operation circuit of a convolutional neural network, a mux 100 for selecting an X array or a + array from input data 10; a multiplier 200 that multiplies the output of the mux 100 and the weight of the neural network; and an adder 300 that adds the output of the multiplier 200 .

In addition, the input data 10 is a 3*3 array, the X array is the input data 10 of diagonal lines and intersections, the + array is the input data 10 of the crosshairs and the crosshairs, and the mux 100 is 2 inputs One output mux 100 is four, and the number of multipliers 200 and weights is five.

In addition, the mux 100 switches the output of the mux 100 for the input data 10 according to the value of the diagonal input data 10 or the cross-hair input data 10 in the input data 10 . The mux 100 may switch the output of the mux 100 according to the level of the sum of the diagonal input data 10 and the cross-hair input data 10 . In addition, the output of the convolutional neural network according to the input data 10 may be fed back to learn weights. In this case, the weight may vary depending on the selection of the X arrangement or the + arrangement of the input data 10 .

In addition, it is a convolutional neural network that uses the Conv-XP pruning device to construct an acceleration circuit that is an arithmetic circuit.

When using the Conv-XP pruning apparatus of the convolutional neural network suitable for the acceleration circuit according to the present invention as described above, the number of multipliers 200 can be reduced to make a high-speed circuit.

In addition, there is an advantage in that the area of the acceleration circuit in the arithmetic circuit is reduced.

1 is an exemplary diagram illustrating an operation circuit of a conventional convolutional neural network.
2 is an exemplary diagram illustrating another embodiment of a conventional convolutional neural network.
3 is an exemplary diagram illustrating an X arrangement and a + arrangement.
4 is an exemplary diagram illustrating an operation circuit of a convolutional neural network of the present invention.

The terms used in the specification take the form of a compound word having a new meaning in a broad sense by combining words and words having different meanings into one word, the original meaning of the word can be expanded and interpreted. For example, the word convolution and the word neural network are combined to mean a convolutional neural network. In addition, the embodiment presented in the specification is a preferred embodiment for implementation, and in some cases, it is possible to add another configuration or to omit the original configuration. In an embodiment, it may be implemented as including or having a configuration.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

3 is an exemplary diagram illustrating an X arrangement and a + arrangement.

In the 3*3 input data 10, the X array has non-zero input data 10 diagonally, and the + array has non-zero input data 10 cross-haired. In the 3*3 input data 10, an X array or a + array having non-zero input data 10 is constituted by diagonal or crosshairs. The 3*3 input data 10 specified by the X array or the + array is input to the arithmetic circuit of the convolutional neural network. The computational circuit of the convolutional neural network performs multiplication and addition operations by adapting to the X array or the + array.

The pruning technique is to make an X array or a + array from 3*3 input data (10). The pruning technique is essential for accelerating the computational circuit of a convolutional neural network by making only a specific array a non-zero value and excluding the zero value from the computation. Pruning techniques can be used to make convolutional neural networks into high-speed circuits.

The Conv-XP pruning apparatus prunes some coefficients among nine coefficients in one kernel, which is a unit simultaneously processed in FIGS. 1 and 2 . In the conventional pruning technique, it is not determined how many coefficients out of 9 coefficients will be pruned and which coefficients will be 0. On the other hand, in the Conv-XP pruning device, pruning is performed only in two patterns as shown in FIG. 3 . In FIG. 3 , a white cell indicates a coefficient that is pruned to 0, and a gray indicates a coefficient that remains as a non-zero value after pruning. The two patterns resemble the letter X and the symbol +, respectively, so they are called Conv-XP pruning.

Compared to the conventional pruning technique, the Conv-XP pruning device prunes the convolutional neural network to fit the acceleration circuit. The Conv-XP pruning device prunes so that 4 coefficients are always 0 for each 3*3 kernel that is simultaneously processed in the acceleration circuit. Therefore, it is guaranteed that a 3*3 kernel can always be processed in one cycle even with 5 multipliers. And since only X array and + array are used, there is a possibility that only two input data can be operands to one multiplier. An operation circuit of the convolutional neural network using these characteristics is shown in FIG. 4 .

The Conv-XP pruning device is applied to each 3*3 kernel, and for each 3*3 kernel, the sum of the absolute values of the coefficients in the X array and the + array is obtained, and then the array with the larger sum is selected. take and zero the coefficients according to the array. After pruning all 3*3 kernels, it learns again while maintaining 0 coefficients.

4 is an exemplary diagram illustrating an operation circuit of a convolutional neural network of the present invention.

In the computation circuit of the convolutional neural network, the Conv-XP pruning device of the convolutional neural network suitable for the acceleration circuit comprises: a mux 100 for selecting an X array or a + array from input data 10; a multiplier 200 that multiplies the output of the mux 100 and the weight of the neural network; and an adder 300 that adds the output of the multiplier 200 .

The input data 10 is a 3*3 array, the X array is the input data 10 of diagonal lines and intersections, the + array is the crosshairs and the cross-dot input data 10, and the mux 100 is 2 inputs 1 output. The mux 100 is four, and the number of multipliers 200 and weights is five.

The mux 100 switches the output of the mux 100 for the input data 10 according to a value of the diagonal input data 10 or the cross-hair input data 10 in the input data 10 . The mux 100 may switch the output of the mux 100 according to the level of the sum of the diagonal input data 10 and the cross-hair input data 10 . In addition, the output of the convolutional neural network according to the input data 10 may be fed back to learn weights. In this case, the weight may vary depending on the selection of the X arrangement or the + arrangement of the input data 10 .

Using the Conv-XP pruning device, a convolutional neural network composing an acceleration circuit, which is a computational circuit, is created.

Another advantage of using the Conv-XP pruning device is that the memory area can be reduced. In the conventional pruning technique, in order to select input data to be multiplied by each non-zero coefficient, the selection signal of the 9-to-1 MUX positioned in front of each multiplier must be stored together. A 4-bit selection signal is required for one MUX, and assuming 5 multipliers, a total of 20-bit selection signal is required.

However, when using the Conv-XP pruning device, there is only a 2-to-1 mux in front of each multiplier because it selects only between the X array and the + array. The length of the selection signal of this mux is sufficient as 1 bit. And since the muxes in front of the 5 multipliers share the same selection signal, 5 non-zero coefficients share one selection signal. Therefore, the selection signal of 20 bits for the 5 multipliers is reduced to 1 bit, and thus the size of the memory storing these selection signals is reduced.

VGG16 ResNet-50 baseline 88.44% 91.14% Conv-XP 89.74% 91.30%

In Table 1, the accuracy before applying the Conv-XP pruning device ('baseline') and when re-learning after applying the Conv-XP pruning device ('Conv-XP') are shown for VGG16 and ResNet-50. Compare. The re-learning was performed for 12 epochs with the learning rate set to 0.001. According to the results in Table 1, it can be seen that the performance of the convolutional neural network does not deteriorate even if the Conv-XP pruning device is applied. In order to compare the improvement in the complexity of the acceleration circuit when using the Conv-XP pruning device, three circuits were designed at RTL (register transfer level) and then synthesized and compared. The first is the conventional structure as shown in Fig. 1 assuming an unpruned convolutional neural network, the second is the structure as shown in Fig. 2, which is a sparse acceleration circuit for a convolutional neural network to which the conventional pruning technique is applied, and the third is a sparse acceleration circuit for a convolutional neural network pruned by the Conv-XP pruning device, as shown in FIG. 4 . In Table 2, ‘No sparsity’, ‘Conventional sparsity’, and ‘Conv-XP’ are indicated respectively. The synthesis library used Global Foundry's 65nm process. In Table 2, a processing element (PE) means a block that can simultaneously process one 3x3 kernel by consisting of 9 or 5 multipliers 200, and it is assumed that there are 16 such PEs.

16 PEs Weight Buffer Index Buffer Activation MUX Miscellaneous Total No sparsity 642826 507042 - - 3199 1153067 Conventional sparsity 357640 281691 81895 25078 56 746360 Conv-XP 357364 281691 5916 6157 47 651175

In the sparse acceleration circuit ('Conventional sparsity' in Table 2) for the conventional pruning technique, the number of multipliers 200 is 9 in comparison to the general acceleration circuit ('No sparsity' in Table 2) that does not consider pruning. As the number of coefficients is reduced to 5 and the number of required coefficients is also reduced from 9 to 5, the area of PE and the counting buffer (Weight Buffer) is reduced. However, the Activation MUX that collects the 9-to-1 MUXs depicted in Figure 2 is added, and the Index Buffer that stores the MUX selection signal is also added. Pruning the convolutional neural network with the Conv-XP pruning device accelerates sparse acceleration. The size of Activation MUX and Index Buffer can be reduced in the circuit. In the existing pruning technique, since there is no pruning pattern, a 9-to-1 MUX that selects one data from all nine input data (10) was required, but Conv-XP prunes with one of two patterns. A 2-to-1 MUX should suffice. Therefore, the size of the Activation MUX block is reduced to about 25%. Also, the bit width of the MUX selection signal is reduced from 4 bits to 1 bit, and since 5 coefficients share one selection signal, it is eventually reduced from 20 bits to 1 bit. As a result, the size of the index buffer was reduced to about 7.2%. Due to these effects, assuming the Conv-XP pruning device, the area of the sparse acceleration circuit is reduced by about 12.8% compared to the conventional sparse acceleration circuit.

Although the above has been described with reference to preferred embodiments of the present invention, those skilled in the art can variously modify and change the present invention within the scope without departing from the spirit and scope of the present invention as set forth in the claims below. You will understand that it can be done.

100: mux 200: multiplier
300: adder

Claims (3)

In the computation circuit of a convolutional neural network,
a mux 100 for selecting an X array or a + array from the input data 10;
a multiplier 200 that multiplies the output of the mux 100 and a weight of the neural network; and
including an adder 300 that adds the output of the multiplier 200,
The input data 10 is a 3*3 array, the X array is input data 10 of diagonal lines and intersections, the + array is input data 10 of crosshairs and cross points, and the mux 100 is Conv-XP pruning device of a convolutional neural network suitable for an acceleration circuit, characterized in that the number of 2 input 1 output mux 100 is 4, and the number of the multipliers 200 and the weight is 5. delete delete

Images