KR102256288B1 - Pruning-based training method and system for acceleration hardware of a artificial neural network - Google Patents

Abstract

Disclosed is a pruning-based training method for acceleration hardware of an artificial neural network. The pruning-based training method for acceleration hardware of an artificial neural network comprises: a step in which a processing unit identifies a SIMD width of a processing element included in an accelerator; a step in which the processing unit initializes the weights of the artificial neural network; a step in which the processing unit groups weights arranged in an input channel and an output channel of the artificial neural network into weight groups according to the identified SIMD width; a step in which the processing unit prunes the plurality of weights included in the artificial neural network by being aware of the weight groups; and a step in which the processing unit updates the plurality of pruned weights. The step in which the processing unit prunes the plurality of weights included in the artificial neural network by being aware of the weight groups comprises: a step in which the processing unit extracts representative weights from each of the weight groups; a step in which the processing unit compares the representative weights with thresholds, respectively; and a step in which the processing unit sets all of the weights belonging to the weight group corresponding to the one of the representative weights to zero when any one of the representative weights is less than a corresponding threshold. Accordingly, by means of the method according to the present invention, it is possible to achieve energy saving effects in the accelerator.

Description

{Pruning-based training method and system for acceleration hardware of a artificial neural network}

The present invention relates to a pruning-based training method and system for accelerating hardware of an artificial neural network, and in particular, a pruning-based training method for accelerating hardware of an artificial neural network to improve computational speed and reduce energy consumption in acceleration hardware. And a system.

Artificial neural networks are used in a variety of artificial intelligence applications such as computer vision, natural language processing, and speech recognition. Artificial neural networks can be divided into a training stage and an inference stage. In the training phase, the parameters of the artificial neural network are learned using the training data. In the inference stage, new input data are applied to the trained artificial neural network, and the prediction result of the artificial neural network is output.

The training operation is performed on the CPU or GPU. Inference operations are performed on an AI accelerator, which is an acceleration hardware specially designed to accelerate artificial intelligence applications.

As the size of artificial neural networks increases in order to perform more complex operations and increase accuracy, energy consumption may increase and bottlenecks may occur.

Korean Patent Application Publication No. 10-2020-0093404 (2020.08.05.)

The technical task to be achieved by the present invention is to solve the conventional problems as described above, and a branch for acceleration hardware of an artificial neural network capable of reducing energy consumption and improving computation speed by reducing the size of the artificial neural network in the training stage of the artificial neural network. It is to provide a stroke-based training method and system.

In the pruning-based training method for acceleration hardware of an artificial neural network according to an embodiment of the present invention, the processing unit calculates the SIMD (Single Instruction Multiple Data) width of a processing element included in an accelerator. Identifying, the processing unit initializing weights of the artificial neural network, the processing unit grouping the weights arranged in the input channel and the output channel of the artificial neural network into weight groups according to the identified SIMD width, The processing unit aware of the weight groups and pruning the plurality of weights included in the artificial neural network, and the processing unit updating the pruned plurality of weights. do.

The processing unit may be a central processing unit (CPU) or a graphical processing unit (GPU).

When the SIMD width is 2, two weights among the plurality of weights form one weight group.

When the SIMD width is 4, four weights among the plurality of weights form one weight group.

When the SIMD width is 8, eight weights among the plurality of weights form one weight group.

The processing unit is aware of the weight groups and pruning the plurality of weights included in the artificial neural network, wherein the processing unit extracts representative weights from each of the weight groups, the The processing unit compares each of the representative weights with each of the threshold values, and when any one of the representative weights is less than a corresponding threshold value, the processing unit performs a weight corresponding to any one of the representative weights. And setting all of the weights belonging to the group to 0.

The processing unit is aware of the weight groups and pruning the plurality of weights included in the artificial neural network, when the other one of the representative weights is greater than a corresponding threshold value, the The processing unit may further include maintaining all weights belonging to a weight group corresponding to the other one of the representative weights.

The weights having 0 in the input channel of the artificial neural network and the output channel of the artificial neural network are continuously present as long as they correspond to the identified SIMD width.

The processing element includes a weight register.

The weight register is implemented as a matrix.

Weights arranged in the output channel of the artificial neural network are sequentially inserted along the rows of the matrix. Weights arranged in the input channel of the artificial neural network are sequentially inserted along the column of the matrix.

In the pruning-based training method for the acceleration hardware of the artificial neural network, the processing unit includes weights of the artificial neural network and the threshold value based on a slope of a loss function and a slope of the loss function related to the threshold values. It may further include the step of updating them.

A pruning-based training system for acceleration hardware of an artificial neural network according to an embodiment of the present invention includes a memory for storing instructions and a processing unit for executing the instructions.

The instructions identify the SIMD width of the processing element included in the accelerator, initialize the weights of the artificial neural network, and group the weights arranged in the input channel and the output channel of the artificial neural network into weight groups according to the identified SIMD width. The plurality of weights included in the artificial neural network are pruned by being conscious of the weight groups, and the plurality of pruned weights are updated.

The processing unit may be a central processing unit (CPU) or a graphical processing unit (GPU).

When the SIMD width is 2, two weights among the plurality of weights form one weight group.

When the SIMD width is 4, four weights among the plurality of weights form one weight group.

When the SIMD width is 8, eight weights among the plurality of weights form one weight group.

The command for pruning the plurality of weights included in the artificial neural network consciously of the weight groups extracts a representative weight from each of the weight groups, and compares each of the representative weights with each of the threshold values.

When any one of the representative weights is less than a corresponding threshold value, all weights belonging to the weight groups corresponding to any one of the representative weights are set to 0.

The command for pruning the plurality of weights included in the artificial neural network consciously of the weight groups is a weight group corresponding to the other one of the representative weights when the other one of the representative weights is greater than a corresponding threshold value It is implemented to keep all of the weights belonging to the fields.

The weights having 0 in the input channel of the artificial neural network and the output channel of the artificial neural network are continuously present as long as they correspond to the identified SIMD width.

The processing element includes a weight register.

The weight register is implemented as a matrix.

Weights arranged in the output channel of the artificial neural network are sequentially inserted along the rows of the matrix. Weights arranged in the input channel of the artificial neural network are sequentially inserted along the column of the matrix.

The instructions are implemented to update the weights of the artificial neural network and the threshold values based on the slope of the loss function and the slope of the loss function related to the threshold values.

In the pruning-based training method and system for acceleration hardware of an artificial neural network according to an embodiment of the present invention, pruning is not performed in the accelerator after training on the weights of the artificial neural network, but the weight of the artificial neural network. There is an effect that flexible weights can be provided to the hardware structure of the accelerator by performing pruning in consideration of the SIMD (Single Instruction Multiple Data) width of the processing unit included in the accelerator in the training phase of the accelerator.

In addition, the pruning-based training method and system for acceleration hardware of an artificial neural network according to an embodiment of the present invention group the weights arranged in the input channel and the output channel of the artificial neural network, and the SIMD width of the processing unit included in the accelerator. By training the grouped weights according to the accelerator, it is possible to improve the computational speed and reduce the energy consumption.

In order to more fully understand the drawings cited in the detailed description of the present invention, a detailed description of each drawing is provided.
1 is a block diagram of a system according to an embodiment of the present invention.
FIG. 2 is a block diagram of an artificial neural network for explaining the operation of the processing unit shown in FIG. 1.
3 shows input activations and weights of a hidden layer of the artificial neural network shown in FIG. 2.
FIG. 4 shows a 4D weight tensor and a 2D weight matrix shown in FIG. 3.
5 shows a block diagram of the accelerator shown in FIG. 1.
6 shows an internal block diagram of the processing element shown in FIG. 5.
7 is a flowchart illustrating a method of operating the system shown in FIG. 1.
8 shows another internal block diagram of the processing element shown in FIG. 5.
9 shows an internal block diagram of a plurality of processing elements shown in FIG. 5.
10 is a flowchart illustrating another method of operating the system shown in FIG. 1.

Specific structural or functional descriptions of the embodiments according to the concept of the present invention disclosed in the present specification are exemplified only for the purpose of describing the embodiments according to the concept of the present invention, and the embodiments according to the concept of the present invention are It may be implemented in various forms and is not limited to the embodiments described herein.

Since the embodiments according to the concept of the present invention can apply various changes and have various forms, the embodiments will be illustrated in the drawings and described in detail in the present specification. However, this is not intended to limit the embodiments according to the concept of the present invention to specific disclosed forms, and includes all changes, equivalents, or substitutes included in the spirit and scope of the present invention.

Terms such as first or second may be used to describe various constituent elements, but the constituent elements should not be limited by the terms. The terms are only for the purpose of distinguishing one component from other components, for example, without departing from the scope of the rights according to the concept of the present invention, the first component may be referred to as the second component, and similarly The second component may also be referred to as a first component.

When a component is referred to as being "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. It should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle. Other expressions describing the relationship between components, such as "between" and "directly between" or "adjacent to" and "directly adjacent to" should be interpreted as well.

The terms used in the present specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present specification, terms such as "comprise" or "have" are intended to designate the presence of a set feature, number, step, action, component, part, or combination thereof, but one or more other features or numbers. It is to be understood that the possibility of addition or presence of, steps, actions, components, parts, or combinations thereof is not preliminarily excluded.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms as defined in a commonly used dictionary should be construed as having a meaning consistent with the meaning of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in the present specification. Does not.

Hereinafter, the present invention will be described in detail by describing a preferred embodiment of the present invention with reference to the accompanying drawings.

1 is a block diagram of a system according to an embodiment of the present invention. FIG. 2 is a block diagram of an artificial neural network for explaining the operation of the processing unit shown in FIG. 1.

Referring to FIGS. 1 and 2, the system 100 is an electronic device such as a smart phone, a notebook computer, a tablet PC, a PC, or a server capable of improving calculation speed and reducing energy consumption. The system 100 includes a central processing unit (CPU) 10, a graphics processing unit (GPU) 20, a memory 30, and an accelerator 200. The system 100 shown in FIG. 1 represents an embodiment for describing the present invention. Depending on the embodiment, the system 100 may be implemented in various ways. For example, the system 100 may further include an input device (not shown) or a transceiver (not shown). The CPU 10, the GPU 20, the memory 30, and the accelerator 200 are connected to the bus 101. The CPU 10, the GPU 20, the memory 30, and the accelerator 200 exchange commands and data through the bus 101.

The CPU 10 executes instructions for training parameters which are weights and bias vectors of the artificial neural network 103. According to an embodiment, training of the artificial neural network 103 may be performed by the GPU 20.

The memory 30 stores instructions executed in the CPU 10 or GPU 20. In the present invention, the processing unit means the CPU 10 or the GPU 20.

The accelerator 200 performs inference of the artificial neural network 103 after training of the artificial neural network 103 is completed. According to an embodiment, the accelerator 200 may be referred to as an AI accelerator. In addition, according to an embodiment, the accelerator 200 may be referred to as acceleration hardware.

Referring to FIG. 2, the artificial neural network 103 includes an input layer 105, a hidden layer 107, and an output layer 109.

The input layer 105 receives each value. The input layer 105 duplicates each value and transmits each duplicated value to the hidden layer 107. Each of the above values may be referred to as input activations. The memory 30 stores each value.

The hidden layer 107 multiplies each value and weights, and outputs a weighted sum. The weighted sum may be referred to as output activations. The memory 30 stores weights and a sum of weights.

According to an embodiment, the hidden layer 107 may apply the sum of weights to the activation function. In the training phase of the artificial neural network 103, the product of each value and weights is performed by the CPU 10 or the GPU 20. In the inference step of the neural network 103, the product of each value and the weights is performed by the accelerator 200.

The output layer 109 generates an output value by applying the sum of weights received from the hidden layer 107 to an activation function.

Each of the input layer 105, the hidden layer 107, and the output layer 109 includes nodes. Depending on the embodiment, the number of nodes of the input layer 105, the hidden layer 107, and the output layer 109, and the number of layers of the hidden layer 107 may vary. As the number of nodes of the input layer 105, the hidden layer 107, and the output layer 109 and the number of layers of the hidden layer 107 increase, the size of the artificial neural network 103 increases. An increase in the size of the artificial neural network 103 causes an increase in energy consumption and a bottleneck. Therefore, in the training stage of the artificial neural network 103, it is necessary to decrease the artificial neural network 103 in order to increase energy efficiency and operation speed. In addition, as the number of each value and weight of the artificial neural network 103 increases, the required storage space in the memory 30 increases. Therefore, it is necessary to save the storage space of the memory 30 in the training stage of the artificial neural network 103.

3 shows input activations and weights of a hidden layer of the artificial neural network shown in FIG. 2.

2 and 3, input activation may be expressed as a 3D tensor, which is a data structure used in the artificial neural network 103. The 3D tensor of input activation can be expressed as an H x W x C matrix.

The weight can be expressed as a 4D tensor. The weighted 4D tensor can be expressed as an R x S x C x K matrix. In the weighted 4D tensor,'C' represents the input channel. In the weighted 4D tensor,'K' represents the output channel.

Output activation can be expressed as a 3D tensor. The 3D tensor of output activation can be expressed as a P x Q x K matrix. The output activation is created by multiplying and accumulating the input activation and the weight.

FIG. 4 shows a 4D weight tensor and a 2D weight matrix shown in FIG. 3.

Referring to FIG. 4, the 4D weight tensor corresponds to the 4D weight tensor shown in FIG. 3. The weights are arranged in a matrix of 4D tensors. For example, the coefficients W _(1,1,1,1) , W _(1,1,2,1) , and W _(1,1,3,1) are in the first row of the R x S x 1 x 1 matrix. Are arranged. In the coefficient W _(1,1,1,1) , the first 1 represents the first output channel, the second 1 represents the first input channel, the third 1 represents the first column, and the fourth 1 represents the first row. . This is defined to describe an embodiment of the present invention, and the coordinates of the coefficient may be variously defined according to the embodiment. The coefficients W _(1,2,1,1) , W _(1,2,2,1) , and W _(1,2,3,1) are arranged in the first row of the R x S x 2 x 1 matrix . The coefficients W _(1,C,1,1) , W _(1,C,2,1) , and W _(1,C,3,1) are arranged in the first row of the R x S x C x 1 matrix . C means the number of input channels. The coefficients W _(K,1,1,1) , W _(K ,1,2,1), and W _(K,1,3,1) are arranged in the first row of the R x S x C x K matrix . K means the number of output channels.

1 and 4, the CPU 10 or the GPU 20 converts the 4D weight tensor into a plurality of 2D weight matrices in consideration of the input channel and the output channel of the 4D weight tensor. Specifically, the CPU 10 or GPU 20 transforms the 4D weight tensor into a 2D weight matrix so that the weights arranged in the 4D tensor in the input channel direction are sequentially input in the 2D weight matrix in the column direction. In Figure 4, weights arranged in the direction of the input channel in the 4D tensor W _(1,1,1,1) , W _(1,2,1,1) , W _(1,3,1,1) ,... , And W _(1,C,1,1) are arranged in the first column (C1) in the 2D weight matrix. In Figure 4, weights arranged in the direction of the input channel in the 4D tensor W _(2,1,1,1) , W _(2,2,1,1) , W _(2,3,1,1) ,... , And W _(2,C,1,1) are arranged in the second column (C2) in the 2D weight matrix.

In addition, the CPU 10 or GPU 20 transforms the 4D weight tensor into a 2D weight matrix so that the weights arranged in the output channel direction in the 4D tensor are sequentially input in the 2D weight matrix in the row direction. _{In Figure 4, weights W (1,1,1,1)} , W _(2,1,1,1) , ..., and W _{( K,1 ,1, 1} ) arranged in the direction of the output channel in the 4D tensor _{) Are} arranged in the first row (R1) in the 2D weight matrix. _{In FIG. 4, weights W (1,2,1,1)} , W _(2,2,1,1) , ..., and W _(K,2,1,1 ) arranged in the direction of the output channel in the 4D tensor _{) Are} arranged in the second row (R2) in the 2D weight matrix.

In the 4D tensor of the R x S x C x K matrix, weights located in the same input channel and the same output channel belong to different 2D weight matrices. For example, the weight W _(1,1,1,1) belongs to the first 2D weight matrix, and the weight W _(1,1,2,1) belongs to the second 2D weight matrix.

5 shows a block diagram of the accelerator shown in FIG. 1.

1 and 5, the accelerator 200 includes a controller 210, an input activation buffer 220, a weight buffer 230, an output buffer 240, a first register 250, and a second register 260. ), a plurality of processing elements 270, and an accumulator 280.

The controller 210 receives commands for executing the operation of the accelerator 200 from the CPU 10 or the GPU 20. For example, the instructions may include a load instruction, a multiplication instruction, a sum instruction, and a store instruction. When the controller 210 receives the load command, the controller 210 transfers input activations stored in the first register 250 or weights stored in the second register 260 to the plurality of processing elements 270. The first register 250 or the second register 260 is controlled to load. The controller 219 performs operations of the components 220, 230, 240, 250, 260, 270, and 280 of the accelerator 200 according to commands received from the CPU 10 or the GPU 20. Control.

The input activation buffer 220 receives and stores input activations from the memory 30. The weight buffer 230 receives and stores weights from the memory 30.

The output buffer 240 stores a result value output from the accumulator 280 and transmits the result value to the CPU 10, the GPU 20, or the memory 30.

The first register 250 receives and stores input activations from the input activation buffer 220. The second register 260 receives and stores weights from the weight buffer 230. The first register 250 and the second register 260 may be first in first out (FIFO) registers.

Each of the plurality of processing elements 270 performs a multiplication operation that multiplies the input activations received from the first register 250 and the weights received from the second register 260. Partial sums may be output from the plurality of processing elements 270.

The plurality of processing elements 270 may be referred to as a processing element array 271.

The accumulator 280 stores partial sums output from the plurality of processing elements 270 and updates the multiplied weighted sum by adding the partial sums.

According to embodiments, the structure of the accelerator 200 may be implemented in various ways.

6 shows an internal block diagram of the processing element shown in FIG. 5.

5 and 6, all of the plurality of processing elements 270 have the same structure. One of the plurality of processing elements 270 is representatively shown in FIG. 6.

The processing element 270 includes a weight register 271, an input enable register 273, a plurality of multipliers 275, and an adder 277.

The weight register 271 receives weights from the second register 260. The weight register 271 is implemented as a matrix MX for storing weights.

When the SIMD width is 8, the width of the matrix MX may be 8. The height of the matrix MX is greater than 8 and may be less than 16. That is, eight weights may be arranged horizontally of the matrix MX, and eight or more weights may be arranged vertically of the matrix MX.

Referring to FIGS. 4 and 6, the weighted 4D tensor is converted into a 2D weighted matrix in consideration of the input channel and the output channel of the weighted 4D tensor by the CPU 10 or the GPU 20. In the transformed 2D weight matrix, weights are arranged in consideration of the input channel and the output channel of the weight.

The CPU 10 or GPU 20 identifies a SIMD (Single Instruction Multiple Data) width of the processing element 270 included in the accelerator 200. For example, when the SIMD width of the processing element 270 included in the accelerator 200 is 2, the CPU 10 or the GPU 20 determines the SIMD width 2 of the processing element 270 included in the accelerator 200. Can be identified. When the SIMD width of the processing element 270 included in the accelerator 200 is 4, the CPU 10, or the GPU 20 may identify the SIMD width 4 of the processing element 270 included in the accelerator 200. I can.

The CPU 10 or GPU 20 groups the weights arranged in the input channel and the output channel of the artificial neural network 103 into weight groups according to the identified SIMD width. The input channel of the artificial neural network 103 means'C' shown in FIG. 4, and the output channel of the artificial neural network 103 means'K' shown in FIG. 4.

The CPU 10 or GPU 20 groups the weights arranged in the input channel and the output channel of the artificial neural network 103 into weight groups in the 2D weight matrix transformed according to the identified SIMD width. For example, when the SIMD width is 2, the two weights form one group. When the SIMD width is 4, 4 weights form a group.

When the SIMD width is 8, the weights can be divided by 8. Weights are arranged in the order of the output channels in the rows of the transformed 2D weight matrix. In FIG. 6, the weight group GH1 has eight weights (W _(1,1,1,1) , W _(2,1,1,1) ,..., and W _{(8,1,1,1). )} ) Can be included. The weight group GH2 has eight weights (W _(9,1,1,1) , W _(10,1,1,1) ,..., and W _(16,1,1,1) ). Can include.

The weight group GH3 has eight weights (W _(17,1,1,1) , W _(18,1,1,1) ,..., and W _(24,1,1,1) ). Can include.

Weights are arranged in the order of the input channels in the columns of the transformed 2D weight matrix. The weight group (GV1) has eight weights (W _(1,1,1,1) , W _(1,2,1,1) ,..., and W _(1,8,1,1) ). Can include. The weight group (GV2) has eight weights (W _(1,9,1,1) , W _(1,10,1,1) ,..., and W _(1,16,1,1) ). Can include. The weight group (GV3) has eight weights (W _(1,17,1,1) , W _(1,18,1,1) ,..., and W _(1,24,1,1) ). Can include.

The weights grouped under the control of the CPU 10 or the GPU 20 are stored in the matrix MX included in the weight register 271 in consideration of an input channel and an output channel. For example, in the first row R1 of the matrix MX, the weights W _(1,1,1,1) , W _(2,1,1,1) ,..., and W _(8,1, The weight group GH1 including _{1,1)) is stored.} The first column (C1) of the matrix (MX) has weights (W _(1,1,1,1) , W _(1,2,1,1) ,..., and W _{(1,8,1,1) )} ) and a weight group (GV1) including) is stored. By storing the grouped weights in the matrix MX included in the weight register 271 in consideration of the input channel and the output channel, more weights can be stored in the weight register 271 than before. Since more weights are stored in the weight register 271 than before, more partial sum operations can be performed at one time. That is, the calculation speed can be improved.

Weights (W _(1,1,1,1) , W _(2,1,1,1) ,... , And W _(8,1,1,1) ) are inserted in order. The number of inserted weights (W _(1,1,1,1) , W _(2,1,1,1) ,..., and W _(8,1,1,1) ) is equal to the SIMD width .

Weights (W _(1,1,1,1) , W _(1,2,1,1) , ... , And W _(1,8,1,1) ) are inserted in order. The number of inserted weights (W _(1,1,1,1) , W _(1,2,1,1) , ..., and W _(1,8,1,1) ) is equal to the SIMD width . According to the SIMD width of the accelerator 200, the weights arranged in the input channel C and the output channel K of the artificial neural network 103 are grouped and stored in the weight register 271, thereby improving the computation speed in the accelerator 200. Reduction in energy consumption can be achieved.

Input activation register 273 stores input activations.

Each of the plurality of multipliers 275 multiplies the weight output from the weight register 271 and the input activation output from the input activation register 273. When the SIMD width is 8, the plurality of multipliers 275 is 8. Eight multiplications are performed simultaneously according to one instruction. Depending on the embodiment, the number of the plurality of multipliers may vary. Values output from the plurality of multipliers 275 are input to the adder 277. Adder 277 generates a partial sum. The partial sum generated by the adder 277 may be stored in the matrix MX included in the weight register 271.

7 is a flowchart illustrating a method of operating the system shown in FIG. 1. Specifically, FIG. 7 is a flowchart illustrating a training method of the artificial neural network shown in FIG. 2.

1 to 7, the processing unit identifies the SIMD width of the processing element 270 included in the accelerator 200 (S100). The processing unit is the CPU 10 or GPU 20. When the SIMD width of the processing element 270 included in the accelerator 200 is 8, the processing unit may access the accelerator 200 and identify the SIMD width 8.

The processing unit initializes the weights of the artificial neural network 130 (S200). The plurality of weights may be arbitrarily initialized. Also, according to an embodiment, the plurality of weights may be initialized to predetermined values obtained through different artificial neural networks.

The processing unit groups the weights arranged in the input channel and the output channel of the artificial neural network 103 into weight groups according to the identified SIMD width (S300). That is, the processing unit divides the weights arranged in the input channel and the output channel of the artificial neural network 103 into weight groups according to the identified SIMD width. The input channel of the artificial neural network 103 is represented by'C' in FIG. 3. The output channel of the artificial neural network 103 is represented by'K' in FIG. 3. The weights arranged in the input channel of the artificial neural network 103 refer to weights (W _(1,1,1,1) , W _{(1,2,1) sequentially arranged along the input channel (C) in FIG. 4. 1) means} ,..., and W _{( 1,C,1 ,1)} ). The weights arranged in the output channel of the artificial neural network 103 are weights (W _(1,1,1,1) , W _{(2,1,1) sequentially arranged along the output channel K in FIG. 1) means} , ..., and W _{( K,1 ,1,1)} ). When the SIMD width is 2, the two weights form a group.

When the SIMD width is 4, 4 weights form a group.

When the SIMD width is 8, 8 weights (e.g., (W _(1,1,1,1) , W _(1,2,1,1) ,..., and W _{(1,8,1, 1)} ) forms a group.

In order to group the weights arranged in the input channel and the output channel of the artificial neural network 103 according to the identified SIMD width, the processing unit considers the input channel and the output channel of the 4D weight tensor, and the 4D weight tensor has a plurality of 2D weights. Can be converted into matrices. Specifically, the processing unit may transform the 4D weight tensor into a 2D weight matrix so that weights arranged in the input channel direction in the 4D tensor are sequentially input in the column direction in the 2D weight matrix. In addition, the processing unit transforms the 4D weight tensor into a 2D weight matrix so that the weights arranged in the output channel direction in the 4D weight tensor are sequentially input in the 2D weight matrix in the row direction.

The processing unit may group the weights into weight groups according to the SIMD width in the transformed plurality of 2D weight matrices. For example, assuming that the SIMD width is 8, the processing unit includes eight weights (W _(1,1,1,1) , W _{( 1,2,1,1)} ,..., and W _(1,8,1,1) ) can be formed into one weight group. In addition, the processing unit includes eight weights (W _(1,1,1,1) , W _(1,2,1,1) ,. .., and W _(1,8,1,1) ) can be formed into one weight group. In a similar way, the weights arranged in the 2D weight matrices are grouped.

The processing unit is aware of the grouped weights and prunes the plurality of weights included in the artificial neural network 103 (S400). Being conscious means taking the grouped weights into account when performing a pruning operation.

Pruning aims to reduce the number of connections of the artificial neural network 103 by removing weights that are not important in the artificial neural network 103. Unimportant connections can be removed by setting the weights of the artificial neural network 103 to 0. The specific operation of pruning is as follows.

The processing unit extracts each of the representative weights from each of the grouped weights.

Each of the representative weights may be randomly extracted from among the grouped weights. For example, from the grouped weights (W _(1,1,1,1) , W _(1,2,1,1) ,..., and W _(1,8,1,1) ) W _(1,1,1,1) ) may be extracted as a representative weight.

In addition, the grouped weights (W _(1,1,1,1) , W _(1,2,1,1) ,..., and W _(1,8,1,1) ) according to an embodiment RMS (Root Mean Square) of is calculated and the RMS value can be extracted as a representative weight.

In addition, the grouped weights (W _(1,1,1,1) , W _(1,2,1,1) ,..., and W _(1,8,1,1) ) according to an embodiment The average of is calculated and the average value can be extracted as a representative weight.

In addition, the grouped weights (W _(1,1,1,1) , W _(1,2,1,1) ,..., and W _(1,8,1,1) ) according to an embodiment The minimum value of (eg, W _(1,2,1,1) ), or the maximum value (eg, W _(1,8,1,1) ) may be extracted as a representative weight.

The processing unit compares each of the representative weights with each of the threshold values. Each of the threshold values may be set arbitrarily.

When any one of the representative weights is less than a corresponding threshold value, the processing unit sets all of the grouped weights to 0 to remove all of the grouped weights. For example, the grouped weights (W _(1,1,1,1) , W _(1,2,1,1) ,..., and W _(1,8,1,1) ) are all zeros. Can be set. The setting to 0 may be set as a product of a weight matrix and a sparse matrix. A sparse matrix is a matrix containing many zero values. Therefore, weights having 0 in the input channel (C) of the artificial neural network 103 and the output channel (K) of the artificial neural network (e.g., W _(1,1,1,1) , W _{(1,2,1, 1)} ,..., and W _(1,8,1,1) ) may be present continuously as long as they correspond to the identified SIMD width (eg, 8). In addition, in the matrix MX implemented in the processing element 270, values of weights arranged in one row may be all zeros, or values of weights arranged in one column may be all zeros. The number of zeros exists as much as the SIMD width. For example, when the SIMD width is 8, the number of 0s is 0. Weights having 0 in the matrix MX implemented in the processing element 270 according to an embodiment (eg, W _(1,1,1,1) , W _(1,2,1,1) ,... , And W _(1,8,1,1) ) may be excluded. That is, weights having 0 in the matrix MX implemented in the processing element 270 (eg, W _(1,1,1,1) , W _(1,2,1,1) ,..., and W _(1,8,1,1) ) may not be found.

Conversely, when the other one of the representative weights is greater than a corresponding threshold value, the processing unit performs all of the grouped weights (e.g., W _(1,1,1,1) , W _{(2,1,1, 1)} Keep, ..., and W _(8,1,1,1) ). Holding means that all of the grouped weights (e.g., W _(1,1,1,1) , W _(2,1,1,1) , ..., and W _(8,1,1,1) ) Means do not change. That is, the multiplication operation of the weight matrix and the sparse matrix is not performed.

The processing unit updates the plurality of pruned weights (S500). Non-zero weights are updated.

Referring to FIG. 2, a processing unit such as CPU 10 or GPU 20 propagates activation data from training data through artificial neural network 103. That is, the processing unit inputs training data to the input layer 105 and outputs activation data through the hidden layer 107 and the output layer 109.

The processing unit calculates a value of a loss function based on the activation data and target data. The value of the loss function may be calculated using Mean Squared Error (MSE) or cross-entropy between the activation data and target data.

The processing unit includes a value of the loss function to calculate a gradient of the loss function related to the weights of the artificial neural network 103 and a gradient of the loss function related to the threshold values. Backpropagating (backpropagating). The backpropagation operation includes calculating a gradient of the loss function related to weights of the artificial neural network 103 and a gradient of the loss function related to threshold values. The backpropagation aims to minimize the value of the loss function by adjusting the weights and threshold values.

The loss function related to the weights refers to a change in the value of the loss function according to the change of the weights. The loss function related to the threshold values means a change in the value of the loss function according to the change of the threshold values. The processing unit updates the weights of the artificial neural network 103 and the threshold values according to the slope of the loss function related to the weights of the artificial neural network 103 and the slope of the loss function related to the threshold values. . According to the update of the threshold values, all of the specific weight groups may be reset to 0, or the setting to 0 may be canceled.

The processing unit performs structured sparisty regularization on the artificial neural network 103 to avoid overfitting of the artificial neural network 103. Structured sparse normalization means performing a normalization operation in consideration of weight groups. The update operation of the threshold values is performed in a process of performing structured element normalization. The optimization target of the structured sparse normalization may be expressed as the following equation.

[Equation 1]

는 최적화 목표를, 상기

는 문턱값을, 상기

는 가중치들을, 상기

는 분류 손실(classification loss)을, 상기

는 어느 하나의 그룹에 속한 가중치들을 얼마나 패널티를 주기 위해(penalize) 결정하는 페널티 텀(penalty term), 혹은 정규화 파라미터(regularization parameter)를, 상기

는 구조화된 희소 정규화 텀(structured sparisty regularization term을 의미한다. 상기 N은 콘볼루션 층들(convolution layers)의 수를 의미하며, 상기 n은 가중치 텐서의 순서를 의미한다. 구체적으로

는 L1 규범(norm) 또는 L2 규범(norm)일 수 있다. remind

Is the optimization goal,

Is the threshold, above

Is the weights, above

Is the classification loss,

Is a penalty term or a regularization parameter that determines how much to penalize weights belonging to any one group, the

Denotes a structured sparse regularization term, where N denotes the number of convolution layers, and n denotes the order of weight tensors.

May be an L1 norm or an L2 norm.

)은 최종 손실(overall loss)을 최소화하기 위해 업데이트된다. 상기 구조화된 희소 정규화 텀(

)은 손실을 최소화하는 방향으로 학습된다. 상기 문턱값(

)은 희소화(sparsity)를 증가시키는 방향으로 업데이트된다. The classification loss (

) Is updated to minimize overall loss. The structured sparse normalization term (

) Is learned in the direction of minimizing the loss. The threshold value (

) Is updated in the direction of increasing sparsity.

The weight groups are groups in which weights arranged in the input channel C and the output channel K of the artificial neural network 103 are grouped in consideration of the SIMD width. The non-significant weight groups are set to 0 in order to remove non-significant weight groups through the structured sparse normalization.

In the present invention, the weights arranged in the input channel C and the output channel K of the artificial neural network 103 are grouped in consideration of the SIMD width, and a plurality of weights included in the artificial neural network 103 are conscious of the grouped weights. By pruning the fields, it is possible to improve the computational speed and reduce energy consumption in the accelerator 200. In the processing element 270 of the accelerator 200, a multiplication operation of input activations and weights is performed. By pruning the weights of the artificial neural network 103 in consideration of the multiplication operation in the processing element 270 of the accelerator 200, it is possible to improve the computation speed and reduce the energy consumption. It is possible to increase the calculation speed and reduce energy consumption by calculating the weights that are consecutively 0 at once. In addition, it is possible to improve the calculation speed and reduce energy consumption by not calculating the weights that are consecutively zero. By pruning in consideration of the SIMD width, which is the hardware structure of the accelerator 200, it is possible to improve the operation speed and reduce the energy consumption. In addition, it is possible to provide flexible weights to the hardware structure of the accelerator 200 by pruning in consideration of the SIMD width, which is the hardware structure of the accelerator 200. In the prior art, when the hardware structure of the accelerator 200 is changed by setting weights dependent on the hardware structure of the accelerator 200, there is a problem in that parameters of the artificial neural network 103 cannot be optimized. However, the present invention has the advantage that parameters of the artificial neural network 103 can be flexibly changed even if the hardware structure of the accelerator 200 is changed by setting the parameters of the artificial neural network 103 in consideration of the SIMD width. In addition, it is possible to save the storage space of the memory 30 in the training stage of the artificial neural network 103.

If the weights are divided into groups and pruned, the accuracy of the inference may decrease. Also, if the weights are not divided into groups and pruning is performed, pruning performance is degraded. In the present invention, the weights arranged in the input channel C and the output channel K of the artificial neural network 103 are grouped in consideration of the SIMD width, and a plurality of weights included in the artificial neural network 103 are conscious of the grouped weights. By pruning the fields, you can maintain accuracy and increase performance. In addition, in the present invention, the accuracy of inference can be maintained by pruning in consideration of the input channel C and the output channel K of the artificial neural network 103.

8 shows another internal block diagram of the processing element shown in FIG. 5.

1, 5, and 8, the plurality of processing elements 270 all have the same structure. One of the plurality of processing elements 270 is representatively shown in FIG. 5.

The processing element 270 includes a weight register 271, an input enable register 273, a plurality of multipliers 275, and an adder 277.

The weight register 271 receives weights (eg, W ₁₁ to W ₄₄ ) from the second register 260. The weight register 271 is implemented as a matrix MX for storing weights. In FIG. 8, four weights (W ₁₁ to W ₁₄ , W ₂₁ to W ₂₄ , W ₃₁ to W ₃₄ , or W ₄₁ to W ₄₄ ) form one weight group (G1, G2, G3, or G4). do. Each of the weight groups G1 to G4 is arranged in each row in the matrix MX.

The processing unit extracts representative weights from the weight groups G1 to G4. The processing unit compares each of the representative weights with each of the threshold values of the pruning masks. _{The processing unit sets all of the weights (W 21} to W ₂₄ , or W ₃₁ to W ₃₄ ) belonging to a specific group (eg, G2 or G3) among the weight groups (G1 to G4) according to the comparison. I can.

The input activation register 273 receives input activations (eg, a ₁₁ to a ₄₁ ) from the first register 250. The input activations a ₁₁ to a ₄₁ are implemented as a matrix.

Each of the plurality of multipliers 275 multiplies the weights output from the weight register 271 and the input activations output from the input activation register 273. For example, in the first cycle, the multiplier 275 includes the weights W ₁₁ to W ₁₄ included in the first weight group G1 in the weight register 271 and the first column CL1 in the input activation register 273. The input activations (a ₁₁ to a ₁₄ ) included in are multiplied. The multiplier 275 multiplies the weight (W ₁₁ ) by the input activation (a ₁₁ ). The multiplier 275 multiplies the weight (W ₁₂ ) by the input activation (a ₁₂ ). Adder 275 generates a partial sum.

In the second cycle, the multiplier 275 includes the weights W ₂₁ to W ₂₄ included in the second weight group G2 in the weight register 271 and in the second column CL2 in the input enable register 273. The input activations (a ₂₁ ~a ₂₄ ) are multiplied. Adder 275 produces a partial sum.

In the third cycle, the multiplier 275 includes the weights W ₃₁ to W ₃₄ included in the third weight group G3 in the weight register 271 and in the third column CL3 in the input enable register 273. The input activations (a ₃₁ to a ₃₄ ) are multiplied. Adder 275 produces a partial sum.

In the fourth cycle, the multiplier 275 includes the weights W ₄₁ to W ₄₄ included in the fourth weight group G4 in the weight register 271 and in the fourth column CL4 in the input enable register 273. The input activations (a ₄₁ to a ₄₄ ) are multiplied. Adder 275 produces a partial sum.

If, in the weight register 271, assuming that a specific weight group (G2 or G3) among the weight groups G1 to G4 is a zero group set to 0, the weights W ₂₁ to W ₂₄ and input activations An operation of the product of (a ₂₁ to a ₂₄ ), or an operation of the product of the weights W ₃₁ to W ₃₄ and the input activations a ₃₁ to a ₃₄ may be skipped. This leads to an increase in the computational speed of the accelerator 200.

9 shows an internal block diagram of a plurality of processing elements shown in FIG. 5. In FIG. 9, only some of the processing elements are shown for convenience of description. Also, in FIG. 9, only the weight register is shown in the processing element for convenience of description. 9(a) shows an internal block diagram of a plurality of processing elements before a load balancing operation is applied. 9(b) shows an internal block diagram of a plurality of processing elements after applying a load balancing operation.

1 to 9(a), the processing elements 270-1 to 270-3 include weight registers 271-1 to 271-3. In the weight registers 271-1 to 271-3, a hash block means a zero group. That is, all weights included in the zero group are 0. In the weight registers 271-1 to 271-3, a blank block means a non-zero group. Depending on the embodiment, the number of weights forming one group may vary. For example, four weights may form one weight group.

The processing elements 270 under the control of the controller 210 perform multiplication instructions. The first processing element 270-1 multiplies the weights belonging to the first weight group G1 and input activations according to the multiplication instruction. However, since the first weight group G1 is a zero group, the multiplication operation of multiplying the input activations and weights belonging to the first weight group G1 is skipped. At the same time, the second processing element 270-2 multiplies the weights belonging to the first weight group G1 and input activations according to the multiplication instruction. In addition, the second processing element 270-2 generates a partial sum. In addition, the third processing element 270-3 also multiplies the weights belonging to the first weight group G1 and input activations according to the multiplication instruction. In addition, the third processing element 270-3 generates a partial sum.

After the multiplication operation for the first weight group G1 is finished, the processing elements 270-1 to 270-3 sequentially perform a multiplication instruction for the remaining weight groups G2 to G8.

In the weight register 271-1 of the first processing element 270-1, a first weight group (G1), a third weight group (G3), a fifth weight group (G5), and a sixth weight group (G6) Assume these are zero groups. As the number of zero groups in the weight register 271-1 increases, the operation speed of the accelerator 200 may increase. This is because the calculation of the product of weights and input activations in the zero group can be skipped.

In the weight register 271-2 of the second processing element 270-2, it is assumed that the third weight group G3 and the fifth weight group G5 are zero groups. It is assumed that the sixth weight group G6 in the weight register 271-3 of the third processing element 270-3 is a zero group.

The first processing element 270-1 includes more zero groups than other processing elements 270-2 and 270-3. Accordingly, the first processing element 270-1 may perform multiplication instructions faster than the other processing elements 270-2 and 270-3. The third processing element 270-3 includes the smallest zero group and thus has the slowest operation speed among the processing elements 270-1 to 270-3. However, the first processing element 270-1 must wait until the multiplication instructions of the third processing element 270-3 are completely finished. That is, the computational speed of the accelerator 200 is determined according to the processing element 270-3 including the smallest zero group. Therefore, load balancing is required to reduce this latency.

Load balancing is performed in the training phase in order to improve the computational speed of the accelerator 200 in the inference phase.

1 to 9(b), the processing unit includes weight registers 271-1 and 271-2 of the processing elements 270-1, 270-2, and 270-3 of the accelerator 200. In, and 271-3), an arrangement of weights to which the pruning masks are applied is analyzed. That is, the processing unit counts the number of zero weight groups in which the weights arranged in each column are all 0 in each of the weight registers 271-1, 271-2, and 271-3. The zero weight groups mean zero groups.

For example, the processing unit counts the number (four) of zero weight groups G1, G3, G5, and G6 in the weight register 271-1 of the first processing element 270-1. The processing unit counts the number (two) of zero weight groups G3 and G5 in the weight register 271-2 of the second processing element 270-2. The processing unit counts the number of zero weight groups G6 (one) in the weight register 271-3 of the third processing element 270-3.

The processing unit includes weights arranged in a specific weight register (eg, 271-1) among weight registers 271-1, 271-2, and 271-1 during arbitrary epochs according to the analysis of the arrangement. In order not to update the values, threshold values of the pruning mask corresponding to the weights arranged in the specific weight register (eg, 271-1) are maintained. One epoch means that one data is propagated and backpropagated through the artificial neural network 103.

The processing unit is the processing element having the largest number of zero weight groups among the weight registers 271-1, 271-2, and 271-3 of the processing elements 270-1, 270-2, and 270-3. A weight register (eg, 271-1) of (eg, 270-1) is determined. As a result of analyzing the arrangement, the processing unit determines that zero weight groups are the largest in the weight register 271-1 of the first processing element 270-1. Accordingly, the processing unit includes weights arranged in the first weight register 271-1 during arbitrary epochs in order not to update the weights arranged in the weight register 271-1 of the first processing element 270-1. Maintain the threshold values of the pruning mask corresponding to. Therefore, during arbitrary epochs, the first weight group (G1), the third weight group (G3), the fifth weight group (G5), and the sixth weight group (G6) are zero in the first weight register 271-1. The group is maintained, and the remaining weight groups G2, G4, G7, and G8 in the first weight register 271-1 maintain a non-zero group.

According to an embodiment, among the weight registers 271-1, 271-2, and 271-3 of the processing elements 270-1, 270-2, and 270-3, the number of zero weight groups in a specific weight register is It is determined whether it is greater than the sum of the number of zero weight groups and an arbitrary value in other weight registers. Among the weight registers 271-1, 271-2, and 271-3 of the processing elements 270-1, 270-2, and 270-3, weight groups of a specific weight register (eg, 271-1) The number (e.g., 4) is greater than the sum of the number (e.g., 1) of zero weight groups of the weight register 271-3 of another processing element (e.g., 270-3) and an arbitrary value (e.g., 1) When large, the processing unit maintains threshold values of pruning masks corresponding to weights arranged in the specific weight register (eg, 271-1) during arbitrary epochs.

While not updating the weights arranged in the first weight register 271-1, the processing unit performs more pruning on the weights arranged in the remaining weight registers 271-2 and 271-3. Threshold values of pruning masks corresponding to the weights arranged in the remaining weight registers 271-2 and 271-3 are adjusted. Specifically, the processing unit resets threshold values of pruning masks corresponding to the weights arranged in the remaining weight registers 271-2 and 271-3 to be smaller so that more zero groups are generated.

9(b) shows an internal block diagram of a plurality of processing elements after applying a load balancing operation.

Referring to FIG. 9B, the weights arranged in the first weight register 271-1 are not updated. Zero groups and non-zero groups of the first weight register 271-1 are not changed.

However, the weights arranged in the second weight register 271-2 and the third weight register 271-3 are changed. Threshold values of the pruning masks corresponding to the weights arranged in the weight registers 271-2 and 271-3 are adjusted, and the second weight register 271-2 and the third weight register 271-3 are The included weight groups change. The seventh weight group G7 and the eighth weight group G8 of the second weight register 271-2 change from non-zero groups to zero groups. The second weight group G2, the third weight group G3, and the eighth weight group G8 of the third weight register 271-3 change from non-zero groups to zero groups.

After load balancing, the weight registers 271-1, 271-2, and 271-3 all include the same number of zero groups. Therefore, the latency of the accelerator 200 can be reduced.

According to an embodiment, the number of zero groups included in the weight registers 271-1, 271-2, and 271-3 may not be the same. For example, when the number of zero groups of the first weight registers 271-1 is four, the number of zero groups of the second weight registers 271-2 and the third weight registers 271-3 is 5 It can be a dog. Even when the number of zero groups included in the weight registers 271-1, 271-2, and 271-3 are not the same, the latency of the accelerator 200 may be reduced from the beginning.

10 is a flowchart illustrating another method of operating the system shown in FIG. 1.

1 and 8 to 10, the processing unit applies pruning masks to the weights of the artificial neural network 103 (S10). Specifically, the processing unit groups the weights into weight groups. Each of the pruning masks is a sparse matrix. A sparse matrix is a matrix containing many zero values. Applying the pruning masks means multiplying the weight matrix and the sparse matrix. A specific operation of applying pruning masks to the weights of the artificial neural network 103 is as follows.

The processing unit groups the weights into weight groups. Specifically, the processing unit may divide the weights into weight groups in consideration of channels or filters. For example, the processing unit may divide the weights into weight groups in consideration of weights arranged in an input channel or weights arranged in an output channel.

In addition, the processing unit may group weights into weight groups according to a single instruction multiple data (SIMD) width. For example, when the SIMD width is 2, the processing unit may divide the weights into weight groups such that the two weights form one weight group. When the SIMD width is 4, the processing unit may divide the weights into weight groups such that the four weights form one weight group. When the SIMD width is 8, the processing unit may divide the weights into weight groups such that the eight weights form one weight group.

The processing unit compares each of the weight groups with each of the threshold values of the pruning masks. The pruning masks include corresponding threshold values. Specifically, the processing unit extracts representative weights from the weight groups. The representative weights may be randomly extracted from the weight groups. For example, when any one of the weight groups includes eight weights W1 to W8, the first weight W1 may be arbitrarily extracted as a representative weight. In addition, according to an embodiment, when a weight group includes eight weights W1 to W8, a Root Means Square (RMS) of eight weights may be calculated, and an RMS value may be extracted as a representative weight. Further, according to an embodiment, when a weight group includes eight weights W1 to W8, a minimum value or a maximum value among the eight weights may be extracted as a representative weight.

The processing unit compares each of the representative weights extracted from each of the weight groups with each of the threshold values of the pruning masks. Each of the threshold values may be set arbitrarily. Also, the threshold values may be set differently from each other.

When any one of the representative weights is less than a corresponding threshold value, the processing unit sets all weights belonging to a weight group corresponding to any one of the representative weights to 0. For example, when a representative weight among eight weights W1 to W8, which is one weight group, is smaller than a corresponding threshold, the processing unit selects weights W1 to W8 belonging to the weight group corresponding to the representative weight. Set all to 0. The setting to 0 may be set as a product of a weight matrix and a sparse matrix.

Conversely, when the other of the representative weights is greater than a corresponding threshold value, the processing unit maintains all weights belonging to the weight group corresponding to the other of the representative weights. Maintaining means that not all of the weights W1 to W8 belonging to the weight group are changed. That is, the multiplication operation of the weight matrix and the sparse matrix is not performed.

The processing unit propagates activation data from training data through the artificial neural network 103 (S20). That is, the processing unit inputs training data to the input layer 105 of the artificial neural network 103 and outputs activation data through the hidden layer 107 and the output layer 109.

The processing unit calculates a value of a loss function based on the activation data and target data (S30). The value of the loss function may be calculated using Mean Squared Error (MSE) or cross-entropy between the activation data and target data.

The processing unit is configured to calculate a gradient of the loss function related to the weights of the artificial neural network 103 and a gradient of the loss function related to the thresholds of the pruning masks. The value of the loss function is backpropagated (S40). The backpropagation operation includes calculating a gradient of the loss function related to weights of the artificial neural network 103 and a gradient of the loss function related to threshold values. The backpropagation aims to minimize the value of the loss function by adjusting the weights and threshold values.

The value of the loss function changes according to the weights of the artificial neural network 103. Accordingly, the processing unit calculates the slope of the loss function related to the weights by using the change value of the loss function according to the weights.

The value of the loss function also changes according to the threshold values of the pruning masks. This is because weights may be set to 0 according to the threshold values. Accordingly, the processing unit calculates the slope of the loss function related to the threshold values of the pruning masks by using the change value of the loss function according to the threshold values of the pruning masks.

The processing unit includes weights of the artificial neural network 103 based on the gradient of the loss function related to the weights of the artificial neural network 103 and the gradient of the loss function related to the threshold values of the pruning masks, and the The threshold values of the pruning mask are updated (S50). According to the update of the threshold values, all of the specific weight groups may be reset to 0, or the setting to 0 may be canceled.

The processing unit performs structured sparsity regularization on the artificial neural network 103 to avoid overfitting of the artificial neural network 103. Structured sparse normalization means performing a normalization operation in consideration of weight groups.

The processing unit analyzes an arrangement of weights to which the pruning masks are applied in weight registers 271 of processing elements 270 of accelerator 200. The processing unit performs a pruning mask corresponding to the weights arranged in the specific weight register so as not to update the weights arranged in the specific weight register among the weight registers during arbitrary epochs according to the analysis of the arrangement. Keep the thresholds of.

The processing unit adjusts threshold values of pruning masks corresponding to the remaining weights to update the weights arranged in the remaining weight registers excluding the weights arranged in the specific weight register.

The processing unit trains the artificial neural network 103 according to the adjusted threshold values.

In the present invention, the latency in the accelerator 200 may be reduced by pruning in consideration of the zero weight groups arranged in the weight registers 271 of the processing elements 270 of the accelerator 200.

Although the present invention has been described with reference to an embodiment illustrated in the drawings, this is only exemplary, and those of ordinary skill in the art will understand that various modifications and other equivalent embodiments are possible therefrom. Therefore, the true technical protection scope of the present invention should be determined by the technical idea of the attached registration claims.

100: system;
10: CPU;
20: GPU;
30: memory;
101: bus;
200: accelerator;

Claims (16)

The processing unit includes: identifying a SIMD (Single Instruction Multiple Data) width of a processing element included in an accelerator;
The processing unit initializing weights of an artificial neural network;
The processing unit grouping the weights arranged in the input channel and the output channel of the artificial neural network into weight groups according to the identified SIMD width;
The processing unit aware of the weight groups and pruning the plurality of weights included in the artificial neural network; And
The processing unit includes updating the pruned plurality of weights,
The processing unit is aware of the weight groups and pruning the plurality of weights included in the artificial neural network,
The processing unit extracting representative weights from each of the weight groups;
The processing unit comparing each of the representative weights with each of threshold values; And
When any one of the representative weights is less than a corresponding threshold, the processing unit sets all weights belonging to a weight group corresponding to any one of the representative weights to zero. Pruning-based training method for hardware. The method of claim 1, wherein the processing unit,
A pruning-based training method for accelerating hardware of an artificial neural network, which is a CPU (Central Processing Unit) or GPU (Graphic Processing Unit). The method of claim 1,
When the SIMD width is 2,
Two weights among the plurality of weights form one weight group,
When the SIMD width is 4,
Four weights among the plurality of weights form one weight group,
When the SIMD width is 8,
A pruning-based training method for acceleration hardware of an artificial neural network in which eight weights among the plurality of weights form one weight group. delete The method of claim 1, wherein the processing unit is aware of the weight groups and pruning the plurality of weights included in the artificial neural network,
When the other of the representative weights is greater than the corresponding threshold, the processing unit further comprises maintaining all weights belonging to the weight group corresponding to the other of the representative weights. Pruning-based training method for The pruning-based training for acceleration hardware of an artificial neural network according to claim 1, wherein weights having 0 in the input channel of the artificial neural network and the output channel of the artificial neural network are continuously present as long as they correspond to the identified SIMD width. Way. The method of claim 1, wherein the processing element comprises:
Contains a weight register,
The weight register is
Is implemented as a matrix,
Weights arranged in the output channel of the artificial neural network are sequentially inserted along the row of the matrix,
A pruning-based training method for accelerating hardware of an artificial neural network in which weights arranged in an input channel of the artificial neural network are sequentially inserted along the matrix column. The method of claim 1, wherein the training method based on pruning for acceleration hardware of the artificial neural network,
The processing unit further comprises updating weights and threshold values of the artificial neural network based on a slope of a loss function and a slope of the loss function related to the threshold values. Pruning-based training method. A memory for storing instructions; And
And a processing unit that executes the instructions,
The above commands are:
Identifies the SIMD width of the processing element included in the accelerator,
Initialize the weights of the artificial neural network,
Group weights arranged in the input channel and the output channel of the artificial neural network into weight groups according to the identified SIMD width,
Pruning the plurality of weights included in the artificial neural network by being conscious of the weight groups,
It is implemented to update the plurality of pruned weights,
An instruction for pruning the plurality of weights included in the artificial neural network in consideration of the weight groups,
Extracting a representative weight from each of the weight groups,
Compare each of the representative weights with each of the threshold values,
When any one of the representative weights is less than a corresponding threshold, pruning-based hardware for acceleration of an artificial neural network that sets all weights belonging to the weight groups corresponding to one of the representative weights to 0 Training system. The method of claim 9, wherein the processing unit,
A pruning-based training system for the acceleration hardware of an artificial neural network, which is a CPU (Central Processing Unit) or GPU (Graphic Processing Unit). The method of claim 9,
When the SIMD width is 2,
Two weights among the plurality of weights form one weight group,
When the SIMD width is 4,
Four weights among the plurality of weights form one weight group,
When the SIMD width is 8,
A pruning-based training system for acceleration hardware of an artificial neural network in which eight weights among the plurality of weights form one weight group. delete The method of claim 9, wherein the command for pruning the plurality of weights included in the artificial neural network consciously of the weight groups comprises:
When the other of the representative weights is greater than the corresponding threshold, the pruning-based for the acceleration hardware of the artificial neural network is implemented to maintain all the weights belonging to the weight groups corresponding to the other of the representative weights. Training system. The pruning-based training for acceleration hardware of an artificial neural network according to claim 9, wherein weights having 0 in the input channel of the artificial neural network and the output channel of the artificial neural network are continuously present as long as they correspond to the identified SIMD width. system. The method of claim 9,
The processing element,
Contains a weight register,
The weight register,
Is implemented as a matrix,
Weights arranged in the output channel of the artificial neural network are sequentially inserted along the row of the matrix,
A pruning-based training system for accelerating hardware of an artificial neural network in which weights arranged in an input channel of the artificial neural network are sequentially inserted along the matrix column. The method of claim 9, wherein the instructions are:
A pruning-based training system for acceleration hardware of an artificial neural network implemented to update weights of the artificial neural network and the threshold values based on a slope of a loss function and a slope of the loss function related to the threshold values.

Images