KR20200115239A - Apparatus and method for compressing trained deep neural networks - Google Patents

Abstract

Disclosed are a device and a method for compressing a trained deep neural network. According to an embodiment of the present invention, the device comprises: a parameter reduction unit for simplifying a weight matrix describing a trained deep neural network for processing multimedia content; a quantization unit for quantizing the weight matrix reduced by the parameter reduction unit; and an entropy coding unit for scanning the weight matrix quantized by the quantization unit in a predetermined direction, and then sequentially performing entropy-coding for scanned weights to output a bitstream. The parameter reduction unit uses at least one of a pruning method and a low rank approximation method. According to the present invention, a trained deep neural network can be effectively compressed.

Description

Apparatus and method for compressing trained deep neural networks}

The present invention relates to an artificial neural network (ANN) technology, and more particularly, to a compression apparatus and method of a trained deep neural network (DNN) for processing multimedia contents.

Attempts have been made to apply artificial intelligence (AI) in various industries. In particular, the recent artificial intelligence technology uses a neural network (NN), an information processing system having a specific performance in common with a biological neural network, and its performance is greatly improved, and accordingly, the application field is rapidly increasing. have.

This neural network (NN) is also called an'artificial' neural network (ANN). Artificial Neural Networks (ANNs) are distributed parallel information processing models that mimic the behavioral characteristics of animal neurons. ANN has a large number of nodes (also called neurons) that are connected to each other. The ANN has two characteristics: 1) Each neuron computes a weighted input from another adjacent neuron through a specific output function (also called an activation function). 2) The intensity of information transmission between neurons is measured by so-called "weights", and such weights can be adjusted by self-learning of specific algorithms.

ANNs can use different architectures to designate variables and topology relationships included in neural networks. A parameter included in the neural network may be a weight of a connection between neurons as well as activity of neurons. There are feed forward networks and backward propagation neural networks as types of neural network topologies. In the former, nodes within each layer connected to each other in the same layer are supplied to the next stage, which includes some form of'learning rule' that modifies the weight of the connection according to the provided input pattern. The latter allows the propagation of errors in the reverse direction of the weighting adjustment, which is more advanced than the former.

A deep neural network (DNN) responds to a neural network having multiple levels of interconnected nodes and can compactly express a highly nonlinear and highly variable function. Nevertheless, with the number of nodes associated with multiple layers, the computational complexity for the DNN increases rapidly. Until recently, efficient computation methods for learning or training such DNNs have been developed. As the learning speed of DNN increases dramatically, it has been successfully applied to various and complex tasks such as speech recognition, image segmentation, object detection, and facial recognition.

Multimedia content, such as video compression and decompression, is also one of the areas where DNN is being applied. High Efficiency Video Coding (HEVC) as the current next-generation video coding was developed by the joint video project of ITU-T (Video Coding Expert Group) and ISO/IEC MPEG (Video Expert Group) standardization organization and adopted as an international standard. It has been used, and it is known that it is possible to further improve its performance by applying DNN to a new video coding standard such as HEVC. One such attempt is disclosed in Korean Patent Laid-Open Publication No. 10-2018-0052651, "Method and apparatus of neural network-based processing in video coding."

However, the scale of neural networks has been exploding due to rapid development in recent years. Some advanced neural network models will have hundreds of layers and billions of connections. And its implementation is both computation-centric and memory-centric.

Since neural networks are getting bigger, it is very important to make the neural network model small, that is, compression, in order to be applied to devices such as mobile terminals that are limited in storage capacity, processor performance, and communication bandwidth. In particular, in order to apply a neural network model to a video coding application for the production and consumption of multimedia content used as an important application in a mobile terminal, a neural network model of a small size is essential. However, up to now, sufficient research has not been conducted on the technique of compressing the neural network model and restoring the compressed neural network model.

Korean Patent Publication No. 10-2018-0052651, "Method and apparatus for processing based on neural networks in video coding"

One problem to be solved by the present invention is the technology, analysis, and processing of multimedia contents, which can be applied to devices such as mobile terminals, which are limited in storage, processor performance, and communication bandwidth, while minimizing deterioration of the performance of neural networks. It is to provide a training apparatus and method for compressing a deep neural network.

A deep neural network compression apparatus according to an embodiment of the present invention for solving the above problem is a parameter reduction unit for simplifying a weight matrix describing a deep neural network, and the parameter reduction unit. Entropy for scanning a quantization unit for quantizing the reduced weight matrix and a weight matrix quantized by the quantization unit in a predetermined direction, then sequentially entropy coding the scanned weights and outputting them as a bitstream It includes an entropy coding unit, and the parameter reduction unit simplifies the weight matrix by using at least one of a pruning technique and a low rank approximation technique.

According to an aspect of the embodiment, the parameter reduction unit outputs information indicating a technique used for simplification of the weight matrix.

According to another aspect of the embodiment, the low-layer approximation technique is performed by decomposing the original weight matrix of the trained deep neural network into a plurality of lower-dimensional low rank weight matrices. Express. In this case, the parameter reduction unit includes a circulant operator, a rank value, dimension and shape information of the initial weight matrix, and reshaping used in the low-level approximation technique. Among the modes (reshaping mode), one or more can be output.

A method for compressing a deep neural network according to another embodiment of the present invention for achieving the above technical problem includes a parameter reduction step for simplifying a weight matrix describing a trained deep neural network, and the weight matrix reduced in the parameter reduction step. And an entropy coding step of scanning the weight matrix quantized in the quantization step in a predetermined direction and sequentially entropy coding the scanned weights to output a bitstream, and the parameter reduction step , At least one of the pruning technique and the low rank approximation technique is used.

According to an embodiment of the present invention, it is possible to effectively compress a trained deep neural network so that the deep neural network can be applied to the processing of multimedia contents, even in a device such as a mobile terminal, which has limitations in storage, processor performance, and communication bandwidth. Do.

1 is a block diagram showing the configuration of a framework for compression and decompression of a deep neural network.
2 is a block diagram showing a detailed configuration of a computer system in which a device for compressing/decompressing a trained deep neural network for processing multimedia contents according to an embodiment of the present invention is implemented.
3 is a block diagram showing an example of the configuration of a deep neural network compression apparatus according to an embodiment of the present invention.
4 is a diagram illustrating a convolutional neural network (CNN), which is an example of a trained deep neural network according to an embodiment of the present invention.
5 is a flowchart showing an example of a processing procedure according to a pruning technique.
6 is a flowchart showing an example of a matrix decomposition process.
7 is a flow chart showing an example of a uniform quantization process.
8 is a flowchart showing an example of an adaptive quantization process.
9 is a block diagram showing an example of a configuration of an apparatus for reconstructing a deep neural network according to an embodiment of the present invention.

Hereinafter, preferred embodiments and examples of the present invention will be described with reference to the drawings. However, the following embodiments and examples are merely illustrative of preferred configurations of the present invention, and the scope of the present invention is not limited to these configurations. And in the following description, the hardware configuration and software configuration of the device, processing flow, manufacturing conditions, size, material, shape, etc. are not intended to limit the scope of the present invention to this, unless specifically stated otherwise. .

Neural network compression and restoration system

1 is a block diagram showing the configuration of a framework for compression and decompression of a deep neural network. 1, a framework for compressing and reconstructing a neural network includes a parameter reduction module (Part 1), a parameter approximation module (Part 2), a reconstruction module (Part 3), It includes an encoding module (Part 4) and a decoding module (Decoding module, Part 5). In FIG. 1, it is assumed that the original neural network model (Original NN Model) and the reconstructed neural network model (Reconstructed NN Model) are configured in the same model format.

The process in the parameter reduction module (Part 1) is performed from the viewpoint of pre-processing and is an arbitrary process. Therefore, the original neural network model (Original NN Model) may be directly input to the parameter approximation module (Part 2) without going through the process in the parameter reduction module (Part 1). However, in the case of passing through the parameter reduction module (Part 1), information (metadata, etc.) related to the execution of the corresponding module is generated, and may be compressed and transmitted to reconstruct a neural network. For example, information (flags) indicating whether to perform the pruning technique and/or the matrix decomposition technique, as well as information related to the execution of the technique, are metadata, which may be output of the parameter reduction module (Part 1). It will be described later. In addition, the outputted information may be transmitted to the reconstruction module (Part 3) and/or the decoding module (Part 5) for restoration. A detailed process performed in the parameter reduction module (Part 1) will be described later.

In the parameter approximation module (Part 2), a parameter approximation technique is applied to the extracted parameter tensors. The parameter approximation technique includes, for example, techniques such as quantization, transformation, and prediction. As an example of a process performed in the parameter approximation module (Part 2), a quantization process will be described later.

In the reconstruction module (Part 3), the approximated parameter tensor generated by the parameter approximation module (Part 2) is restored. In general, an approximate parameter tensor can be reconstructed using metadata information reconstructed from a bitstream. The input to the reconfiguration module (Part 3) has the same format as the output of the parameter approximation module (Part 2). And the output from the reconfiguration module (Part 3) has the same format as the input of the parameter approximation module (Part 2).

The encoding module (Part 4) uses an entropy encoding technique, which will be described later. In addition, the decoding module (Part 5) performs lossless decoding on the input bitstream. The input of the decoding module (Part 5) is a bitstream specified in the encoding module (Part 4), and in the case of an empty dictionary without items, it has the same format as the input of the encoding module (Part 4). The output of the decoding module (Part 5) has the same format as the input of the parameter approximation module (Part 2) and has a dictionary format. However, the output of the decoding module (Part 5) must be the same as the parameters, metadata, and int_parameter existing in the output of the parameter approximation module (loseless coding).

FIG. 2 is a block diagram showing a detailed configuration of a computer system 100 in which an apparatus for compressing or reconstructing a trained deep neural network according to an embodiment of the present invention is implemented.

Referring to FIG. 2, the computer system 100 includes one or more processors 110, an input/output device interface 120, a network interface 130, an interconnector (BUS, 140), a memory 150, and a storage 160. Includes. The computer system 100 may be one specific device configured as a single computing device, or may be a set of multiple devices including one or more processors and one or more associated memories.

The processor 110 fetches and executes the programming instructions stored in the memory 150 or the storage 160. Similarly, the processor 110 stores or retrieves application data in the memory 150. The input/output device interface 120 is for connecting the input/output device 12 such as a keyboard, a display, and a mouse device to the computer system 100. The network interface 130 communicates with an external network such as an internal network (intranet), the Internet, or a wireless communication network through wired or wireless communication, and transmits data through the data communication network 14.

The interconnector 140 performs a function of transmitting programming commands and application data between the processor 110 and the input/output device interface 120, the storage 160, the network interface 130, and the memory 150. do. The interconnector 140 may be one or more buses. The processor 110 may be implemented as a single central processing unit (CPU) or as a single CPU having multiple CPUs, and multiple processing cores in various implementations. According to one aspect, the processor 110 may be a digital signal processor (DSP).

The memory 150 generally includes a random access memory such as a static random access memory (SRAM), a dynamic random access memory (DRAM), or a flash. Storage 160 is generally a hard disk drive, solid state device (SSD), removable memory card, optical storage, flash memory device, such as connections to a network attached storage (NAS) or a storage area device (SAN). Includes non-volatile memory.

The computer system 100 may include one or more operating systems (OS) 164. A part of the operating system 164 may be stored in the memory 150 and a part of the operating system 164 may be stored in the storage 160. Alternatively, the entire operating system 164 may be stored in the memory 150 or may be stored in the storage 160. The operating system 164 provides an interface between various hardware resources such as the processor 110, the input/output device interface 110, the network interface 130, and the like. In addition, the operating system 164 provides common services such as a time function for an application program.

The deep neural network compression device 152 compresses a trained deep neural network for describing, analyzing, and processing multimedia contents, for example, a deep convolutional neural network, and outputs the compressed data as a bitstream. The technology, analysis, and processing of multimedia contents by the deep neural network is to compress and restore motion pictures using deep neural network technology, facial, iris, fingerprint, voice recognition, and motion detection for security, etc., and diagnosis or analysis of diseases. , Including various applications using multimedia contents such as moving pictures, still images, and audio such as autonomous driving.

For example, an apparatus for compressing or reconstructing a video or a coding tool constituting the same is a representative field to which processing of multimedia contents by a deep neural network can be applied. According to this, the deep neural network compression device 152 is a means for compressing the deep neural network of the learned or trained coding tool and describing it in a compatible format. That is, the compressed deep neural network resulting from the deep neural network compression device 152 corresponds to an interoperable compressed representation of neural networks of the coding tool neural network.

In order to compress the trained deep neural network, the deep neural network compression device 152 performs a series of processes including parameter reduction, quantization, and entropy coding for the trained deep neural network. Thus, the encoded bitstream is output. According to the present embodiment, a pruning technique and/or a matrix decomposition technique are used in the parameter reduction process. For example, before parameter reduction by matrix decomposition, some weights constituting the deep neural network may be removed, or a pruning process for simplifying a connection relationship between weights may be additionally performed.

3 is a block diagram showing an example of the configuration of the deep neural network compression apparatus 152 according to an embodiment of the present invention. Referring to FIG. 3, the deep neural network compression apparatus 152 includes a parameter reduction unit 22, a quantization unit 24, and an entropy coding unit 26. As described above, the deep neural network compression apparatus 152 compresses a trained deep neural network for analyzing multimedia contents (that is, weights and parameters describing the trained deep neural network) using a predetermined algorithm, and then, Finally, the encoded bitstream is output.

The input to the deep neural network compression device 152 includes various information and parameters describing the trained deep neural network.

First, the input to the deep neural network compression device 152 includes various high-level information. For example, when the corresponding deep neural network is applied as a coding tool for coding a video, the high-level information may include information indicating the type of the deep neural network-based coding tool. There are two types of deep neural network based coding tools. More specifically, there are two types of coding tools based on deep neural networks: a first type coding tool that implements essential functions for both an encoder and a decoder, and a second type coding tool that implements a function essential for only one of the encoder and the decoder. . This is, in image/video coding, some coding tools, i.e., the first type coding tool, are required for both the encoder and the decoder, and the rest of the coding tools, i. This is because it is a function to be used. For example, in the video coding process, the in-loop filtering process corresponds to the function of the first type coding tool performed at both the encoder and the decoder, but the intra mode prediction process is the function of the second type coding tool performed only at the encoder. And only determined prediction mode information is sent to the decoder. Therefore, two types of DNN-based coding tools must be considered, and the type of these DNN-based coding tools must be indicated as high-level information of the trained deep neural network-based coding tool.

In addition, the high-level information includes information on the overall configuration of the DNN-based multimedia content processing device (hereinafter, referred to as'DNN-based processing device'). More specifically, the high-level information related to the configuration of the DNN-based processing device is a target application viewed from the perspective of basic functions of the neural network such as recognition, classification, generation, and discrimination. (target application) information, information indicating the type of DNN-based processing device, applying a neural network trained to the inference engine when the encoder performs a specific coding process and applying a standardized image or video coding tool Information indicating what is selected from among them, information on customized content type, autoencoder, CNN (Convolutional Neural Network), GAN (Generative Adversarial Network), RNN (Recurrent Neural Network), etc. Basic information about the algorithm of the trained deep neural network based neural network, basic information about training data and/or test data, information about the capability required by the inference engine in terms of memory capacity and computing power, information about model compression, etc. Include.

The input to the deep neural network compression device 152 also includes various parameters describing the trained deep neural network. Various parameters include information about the neural network model, such as weights of kernels, neurons, and connections. Hereinafter, with reference to the architecture of a convolutional neural network (CNN), which is an example of a deep neural network, this will be described in more detail.

4 is a diagram showing an example of a trained deep neural network according to an embodiment of the present invention, in the case of a convolutional neural network (CNN). 4, the CNN 200 is an input layer (imput layer, 210), a convolutional layer (convolutional layer, 215), a subsampling layer (220), a convolutional layer (225), a subsampling layer (230). , A fully connected (FC) layer 235, 240 and an output layer 245. In the example shown in FIG. 4, the input layer 210 is configured to accept a 32×32 pixel image, and the convolution layer 215 generates six 28×28 feature maps from the input layer.

As such, although the configuration of a specific CNN is shown in FIG. 4, more generally, a CNN includes one or more convolutional layers each having a sub-sampling step, and one or more fully connected layers. layer). In general, the illustrated CNN architecture was devised to use a two-dimensional structure of an input image. For example, CNN achieves this by using local connections and connected weights followed by some form of pooling. In general, compared to all-connected networks with a similar number of hidden units, CNNs are easier to train and tend to have fewer parameters.

In general, the CNN includes a convolutional layer and a subsampling layer, followed by a full connection layer. According to an embodiment, the CNN receives an image of x×y×z as an input in the convolutional layer, where x and y represent the height and width of the image, respectively, and z represents a channel in the image. For example, an RGB image has a z=3 channel. The convolutional layer may include a filter (kernel) of size a×b×c, where a×b is less than x×y and c is less than or equal to z. In general, the size of filter k results in a locally linked structure, which is convolved with the image to produce k feature maps. In addition, each feature map is subsampled over adjacent regions of various sizes.

When the deep neural network is a CNN, the neural network model information includes parameter type information, parameter dimension information, and parameter index information. The parameter type information has a value corresponding to any one of five strings along with the name of the parameter tensor defined by the model structure. The five strings are'Convolution Layer Weight','All Connection Layer Weight','Convolution Layer Bias','All Connection Layer Bias' and'Other'. The parameter dimension information has the name of the parameter tensor defined by the model structure and a value corresponding to the dimension of the corresponding parameter tensor. The parameter index information has the name of the parameter tensor defined by the model structure and a value corresponding to the index of the corresponding parameter tensor.

3, the parameter reduction unit 22 performs a process of reducing a set of parameters describing the deep neural network. The parameter reduction unit 22 may use a sparsity technique such as pruning and/or a matrix decomposition technique. The sparse technique refers to simplifying the frequency of the weights by setting the weights to '0' or by cutting the connection of the neural network. And the matrix decomposition technique refers to the decomposition of the neural network's parameter tensor, that is, the weight matrix into smaller matrices.

The parameter reduction unit 22 continuously applies the sparse technique and the matrix decomposition technique (e.g., first sparse the weights, then decompose the matrix of sparse weights) or only one of the two techniques. You may. In addition, information indicating the method applied by the parameter reduction unit 22 may be output and included in the compressed bitstream.

By reducing the set of parameters describing the deep neural network by the parameter reduction unit 22, the deep neural network compression device 100 as well as a device that reconstructs and uses the deep neural network compressed thereby, such as a mobile device, can perform multiplication operation. (multiplication) and memory load can be reduced. However, since reducing the set of parameters corresponds to a lossy process, the performance of the restored deep neural network inevitably degrades to some extent. Therefore, it is preferable that the parameter reduction unit 22 can reduce the size of the weight matrix while minimizing performance degradation of the deep neural network to be restored.

Typically, in the compression process of the trained deep neural network, the parameter reduction process is performed from the viewpoint of preprocessing of the quantization process, and is an arbitrary process. Accordingly, input data (parameters describing the initial deep neural network model) may be directly input to the quantization unit 24 without passing through the parameter reduction unit 22. However, when input data is input to the parameter reduction unit 22, information (metadata, etc.) related to the execution of the parameter reduction process is generated, and may be encoded together with other data for neural network restoration and included in the bitstream. . For example, information (flags) indicating whether or not the pruning technique and/or the matrix decomposition technique is performed, as well as information related to the execution of the technique, may be output of the parameter reduction unit 22 as metadata. It will be described later.

The input data to the parameter reduction unit 22 are parameters and architecture (structure definition) that describe the neural network specified by a specific deep learning framework. And the output data from the parameter reduction unit 22 is also the same as the input data. There is no particular limitation on the format for describing input data and output data, but Keras/Tensorflow or Pytorch may be applied.

In the'spreading or pruning process', which is one process that can be performed in the parameter reduction unit 22, the learned deep neural networks are made to have a reduced parameter size. In pruning, the importance of the weight in each unit, such as one weight or channel unit, is obtained, and the weight of which importance is lowered is set to '0' or the connection relationship is removed (the parameter value is set to 0). In addition, removing the connection relationship may be performed in a larger unit such as a filter unit, as well as disconnecting the connection by one node or weight.

In this pruning process, a problem arises in setting a good threshold so that each layer of neural networks prunes possible networks while maintaining the original performance. For neural networks composed of multiple layers, particularly considering that a threshold value of one layer may be dependent on threshold values of another layer, a brute force to find a threshold value may not be practical. Also, pruning may require retraining of the network to restore its original performance. It can take a significant amount of time to confirm that the pruning process is efficient. As described herein, in various embodiments, automatic selection of thresholds along with methods for retraining the network may be used to prun the neural network to reduce parameters.

In order to cut out neurons with weak responses or unnecessary parameters in the neural network, the criteria for determining the importance of the parameters can be changed according to the algorithm. This criterion may be defined as an objective function, as shown in FIG. 5. Referring to FIG. 5, the pruning technique may proceed as follows: data type decision -> object function decision & parameter input -> pruning a decision value ( pruning decision value) -> pruned NN model.

In the data type determination process, it is determined which units (individually or channel units) are to be pruned. And in the process of determining the objective function and inputting parameters, the objective function is defined with the importance of parameters. Parameters for defining the objective function include a weight value threshold, a scaling factor, and a gradient. And external parameters include pruning rate and/or hyperparameter for re-training, although arbitrary.

The pruning technique can be largely divided into two methods depending on whether or not it is learned. The first method does not require an additional learning process, and performs the pruning technique based on the weights that have already been learned. For example, a threshold of weights is set in advance, the importance of weights is determined accordingly, and then pruning is performed. The second method requires an additional learning process, and in this case, the hyperparameter and learning data required for learning must be input. In addition, the pruning technique can additionally set the compression rate as well, and the objective function represents the method of the pruning technique. For example, the second pruning method is a method of learning only a scaling factor while maintaining the learned weight, and a method of learning only a threshold that is as close as possible to a specified task-based loss. There is this.

In connection with such a pruning technique, the above-described parameters, a set threshold, etc. may be output as metadata for reconstructing a neural network, and as metadata for reconstructing a neural network. For example, mask information (information indicating a position with a weight of 0) and/or threshold information additionally required in the pruning technique may be output as metadata of a pruning or sparse process. In addition, there is information on whether the number of parameters actually decreased during the process (e.g., a flag indicating whether the dimension value of the reconstruction neural network model and the dimension value of the neural network model before compression are the same). It can be output as metadata of the striking or sparing process.

In the'matrix decomposition process', another process that can be performed in the parameter reduction unit 22, one weight matrix, which is an initial weight matrix, is converted into N (N is an integer greater than or equal to 2) low-dimensional weight matrix. Disassemble. As a result, N indexes are created for each index of one layer, and a model different from the original model is created. Accordingly, not only can the number of weight parameters be reduced, but also a runtime reduction effect can be expected. This matrix decomposition technique is generally applicable to a fully-connected layer and a convolution layer.

An example of a matrix decomposition technique is a low rank (LR) approximation process or a low displacement rank (LDR) approximation process. According to the lower rank approximation process or the lower displacement rank approximation process, the weights of a neural network expressed in a multidimensional matrix form are expressed in a plurality of lower-dimensional matrices, such as a row matrix and/or a column matrix, thereby reducing the amount of computation. It is possible.

For example, the lower rank approximation applied to the convolutional layer can be performed as follows: The convolution weight trained in 2D-CNN is of 4D type (filter size, filter size, number of input channels, number of output channels). Equation 1 shows the approximate equation for each channel. The dimension of W_st is expressed as d x d, where d is the filter size of the convolutional layer. In addition, the dimensions of the matrices Us and Vt that can be approximated by the lower rank are expressed by d x R. Here, R is the target value for the lower rank approximation.

According to an example of lower rank approximation, in order to reduce the data amount of weights, a 2D kernel matrix of weights in each convolutional layer may be transformed into two 1D kernel matrices.

Therefore, the purpose of this method is to find (U,V) which gives the minimum difference between W _st and (U _s ×V _t ). The cost function for optimizing filter reconstruction is shown in Equation 2.

According to another example of lower rank approximation, in order to reduce the amount of data of weights, a 2D kernel matrix of weights in each convolutional layer can be transformed into three kernel matrices (e.g., CP (CANDECOMP/PARAFAC) decomposition). ). That is, one convolution weight is decomposed into three weights. The basic formula of this CP decomposition can be expressed by Equation 3.

Here, R denotes the number of components, which is a tensor rank.

In general, the weight of the trained convolutional layer existing in 2D-CNN is 4D type. The convolutional kernel tensor K having a size of T × S × D × D includes T corresponding to the output channel, S corresponding to the input channel, and D corresponding to the kernel size.

In CP decomposition, the convolutional kernel tensor K is approximated by rank R. This can be expressed by the following equation (4).

및

은 각각 크기

및

를 갖는 3개의 컴포넌트이다. here,

And

Each size

And

It is three components with

Therefore, the decomposed result can be mapped as shown in Equation 5 below.

In the case of general convolution, the number of convolution operations should be performed as much as the number of input channels x output channels, but in the case of depthwise convolution, the number of output channels may be performed.

At this time, the cost function for optimizing the filter reconstruction is shown in Equation 6 below.

이다.here,

to be.

According to the lower rank approximation method according to this embodiment, not only can the number of weight parameters be reduced, but also computation speed can be improved.

In general, the outputs in the matrix decomposition process are matrices A and B (here, matrices A and B are matrix factors, which are circulant operators used for matrix decomposition), rank values, integers. Base), a reshaping mode for filter reconstruction (integer-based), and the dimensions and shape of the initial weight matrix.

Among them, the reshaping mode W'may have one or a plurality of modes. As an example, the reshaping mode (W') may have four modes as in Equation 7, which is only exemplary. And, as described above, whether a mode is applied in the matrix decomposition process is reshaping mode information, and must be transmitted for neural network restoration.

Equation 8 describes a weight matrix of a full connection layer for a lower rank approximation technique. In addition, since the matrix decomposition technique can be applied without reshaping as in Equation 4 (refer to Equations 3 to 5), information on whether reshaping is applied may also be included in the metadata.

Equation 9 describes a weight matrix of a convolutional layer for a lower rank approximation technique.

In the lower displacement approximation technique, the decomposition process is the same as the lower rank approximation technique, but the input to the decomposition process is different from the lower rank approximation technique. Equation 10 describes a lower displacement approximation technique.

Here, the f value is set arbitrarily. And it can be seen that in order to reconstruct the neural network model, matrices A and B must also be outputs of the matrix decomposition unit.

The output of the matrix decomposition process may vary depending on whether the corresponding parametric matrix is for a full connection (FC) layer or a convolutional layer. In addition, the output of the matrix decomposition unit may vary depending on whether the applied technique is a low rank approximation technique or a low displacement approximation technique.

An example of this is shown in FIG. 6. If the lower rank approximation technique is applied to all connection layers (LR & FC layers), the output from the matrix decomposition process is information about whether the number of parameters actually decreased during the process (e.g., a neural network model of reconstruction A flag indicating whether the dimension value of is the same as the dimension value of the neural network model before compression), a rank value, Reshape_mode=none, the dimension of the initial matrix, etc. may be included. Instead, when the lower rank approximation technique is applied to the convolutional layer (LR & Conv layers), the output from the matrix decomposition process is information about whether the number of parameters actually decreased during the process (e.g., the neural network of reconstruction). Flag indicating whether the dimension value of the model and the dimension value of the neural network model before compression are the same), rank value, Reshape_mode= 1 (or 2, 3, 4), the dimension of the initial matrix, etc. Can be included. In addition, when the lower displacement approximation technique is applied to the convolution layer (LDR & Conv layers), the output from the matrix decomposition process is information about whether the number of parameters actually decreased during the process (e.g., restoration Flag indicating whether the dimension value of the neural network model of is equal to the dimension value of the neural network model before compression), rank value, Reshape_mode= 1 (or 2, 3, 4), factor A, B, dimensions of the initial matrix, etc. may be included.

According to one embodiment of the matrix decomposition process, if the k-th layer is a convolutional layer, the initial weight matrix is a four-dimensional weight matrix. The dimension of the weight matrix can be decomposed using the number of input channels, the number of output channels, and the size of the 2D filter kernel. Accordingly, the 4D weight matrix may be decomposed into a plurality of 2D weight matrices, and the size and component of the generated 2D matrix may be determined based on the reshaping mode.

With continued reference to FIG. 3, the quantization unit 24 performs quantization on outputs of the parameter reduction unit 22, for example, a plurality of low-dimensional weight matrices. To this end, quantization is performed based on input quantization bits, and a quantized weight is output by extracting a maximum/minimum value (Max/Min vaules). 7 shows an example of a uniform quantization process by the quantization unit 24.

According to an aspect of the present embodiment, in order to reduce a quantization error by considering a distribution of parameters input for quantization, the quantization unit 24 may perform adaptive quantization. In the adaptive quantization process, a compressed integer weight file and a codebook are input, and a quantization level (r _k ) and a quantization region boundary (d _k ) may be expressed by Equation 11.

An example of such an adaptive quantization process is shown in FIG. 8.

In the adaptive quantization process as shown in FIG. 8, if the quantization error is not low enough, non-uniform quantization can be replaced with uniform quantization. According to this, the input includes a configuration file indicating the number of layers. And, if the size of the quantization error is larger than the uniform quantization, uniform quantization may be used instead of non-uniform quantization.

The quantization process shown in FIG. 8 adds/complements the existing quantization process of storing a quantized value (integer, integer) of a neuralnet weight value in a binary format. According to the quantization process shown in FIG. 8, lossy coding may use not only quantization but also other techniques, and may be, for example, a method of dividing coding in a weight value matrix (with good coding efficiency (RD (Estimated by rate distortion), etc.) Determine the segmentation map). In addition, the quantized values of the weight values are generated by entropy coding (arithmetic coding, CABAC, palette, index map coding, etc.), which is lossless coding. In addition, in the process of decoding (or decompression), reconstruction is performed by inputting a binary file (bitstream), etc., at this time, information on whether to perform the neuralnet model reconstruction in its entirety or only part of it may be included. have.

According to an embodiment, the input to the quantization unit 24 is, for example, an item represented by a numpy array of numpy.float32 format, for example an array of weights matrix (generally, in the form of a tensor). It is in the form of a dictionary containing the name of each of them as parameter tensors and keys. The name of each parameter tensor must match the nomenclature specified by each model definition provided by the coordinator of the model (eg, nomenclature in the test model).

The output from quantization unit 24 contains three keys: "parameters", "int_parameters" and "metadata". Here, "parameters" is a float-in which all parameters that are not modified during the approximation process (i.e., parameters irrelevant to the approximation process) are all parameters as values and their respective names are keys as keys. (stored as point) (e.g., layer_type). These can be represented as numpy arrays in the numpy.int32 format, for example. And "int_parameters" are integer-based parameters generated through the approximation process, that is, all parameter tensors that are modified during the approximation process, their respective integer representation as an item, and their respective names It contains a dictionary represented by keys.

"metadata" includes (stores) all metadata created during the approximation process. This metadata includes all necessary information required for decoding/reconstruction of the compressed deep neural network. That is, in the decoding/reconfiguration process, it should be possible to express parameter tensors of all numpy.int32 format of the network in numpy.float32 format, for example. Metadata can be shared by all parameters (global, global) (e.g., a codebook or step_size generated when performing quantization) or by specific parameters (local, local) (e.g. For example, metadata that changes for each layer property like the per_layer parameter is applicable). In the latter case, for the reconstruction process, a separate mapping between parameter names and metadata must be provided. In order to secure flexibility between different approximation techniques, the specific format of metadata can be arbitrarily selected according to the specific technique used. However, to ensure consistency, a dictionary structure is preferred.

The metadata includes, for example, step_size and codebook information generated in the quantization process. In addition, additionally necessary parameters (eg, lambda values when RD is obtained or hyperparameters necessary when learning step_size is obtained) may be expressed as metadata.

3, the entropy coding unit 26 encodes each of the quantized weights and indexes according to a predetermined algorithm (e.g., entropy encoding), and as a result, a bit stream of a compressed deep neural network is output. . According to the present embodiment, there is no particular limitation on a specific process of entropy encoding, and as long as it is known in the art, it may be applied without limitation, unless an algorithm is essentially impossible to apply due to the nature of entropy encoding.

According to an embodiment, the entropy coding unit 26 may identify a data format capable of encoding and a data format that cannot be encoded. All data that the entropy coding unit 26 cannot interpret (understand) is classified as "auxiliary data", and is delivered directly to the bitstream in a serialized manner. The input of the entropy coding unit 26 has the same format as the output of the quantization unit 24. And the output from the entropy coding unit 26 is, for example, a bitstream represented by a numpy array in numpy.float32 format. This bitstream contains information on all items in the input dictionary. Other information such as the key name or dictionary structure need not be encoded into the bitstream.

In the entropy coding unit 26, for example, it is possible to start scanning the weight parameters from the top left. That is, after scanning in a row-first manner from left to right, rows are scanned from top to bottom. Then, the quantized index values are binarized and stored in a binarized form. For example, the quantized index value may be represented by binarization using values of "Sig_flag", "Sign_flag", "AbsGrXFlag", "MaxNumNoRem" and/or "RemAbs". Here, "Sig_flag" indicates whether or not the index value is 0. "Sign_flag" represents the sign of a non-zero index value. "AbsGrXFlag" indicates the size of the index value, and if AbsGrXFlag==1, the corresponding index value is greater than the X value (generally expressed as an exponent of 2). Accordingly, when the corresponding flag is 0 The branches should continue to appear. "MaxNumNoRem" represents AbsGrX to be expressed as the maximum, and "RemAbs" represents the remaining values generated when the AbsGrXFlag value with MaxNumNoRem is 0 in the form of binary representation. At this time, bins representing the remaining values are coded as bypass. For example, if MaxNumNoRem==8, the value of RemAbs should be sent when the index absolute value is 5-8. In the context model, "Sig_flag", "Sign_flag", "AbsGrXFlag" and the like used in the binarization process correspond.

9 is a block diagram showing an example of a configuration of an apparatus 154 for reconstructing a deep neural network according to an embodiment of the present invention. Referring to FIG. 9, the deep neural network restoration apparatus 154 includes an entropy decoding unit 32 and an inverse quantization unit 34. Depending on the embodiment, the deep neural network restoration apparatus 154 may further include a parameter reinforcement unit 36. As described with reference to FIG. 1, the decoding process in the deep neural network decompression apparatus 154 corresponds to the encoding process in the deep neural network compression apparatus 152 described with reference to FIG. 3. That is, the deep neural network restoration apparatus 154 restores the deep neural network by restoring the data in the opposite direction to the encoding process, based on the metadata included in the input bitstream.

As described above, the above description is merely an example and should not be construed as being limited thereto. The technical idea of the present invention should be specified only by the invention described in the claims to be described later, and all technical ideas within the scope equivalent thereto should be interpreted as being included in the scope of the present invention. Therefore, it is obvious to those skilled in the art that the above-described embodiments can be modified and implemented in various forms.

Claims (5)

A parameter reduction unit for simplifying a weight matrix describing a deep neural network;
A quantization unit for quantizing the weight matrix reduced by the parameter reduction unit; And
And an entropy coding unit for scanning the weight matrix quantized by the quantization unit in a predetermined direction and then sequentially entropy coding the scanned weights to output a bitstream,
The parameter reduction unit simplifies the weight matrix by using at least one of a pruning technique and a low rank approximation technique.
The method of claim 1,
The parameter reduction unit outputs information indicating a technique used for simplification of the weight matrix.
The method of claim 1,
The low-level approximation technique is characterized in that expressing by decomposing the original weight matrix of the trained deep neural network into a plurality of lower-dimensional low rank weight matrices. Deep neural network compression device.
The method of claim 3,
The parameter reduction unit includes a circulant operator, a rank value, dimension and shape information of the initial weight matrix, and a reshaping mode used in the low layer approximation technique. Among them, a compression apparatus for a trained deep neural network, characterized in that outputting one or more.
A parameter reduction step for simplifying a weight matrix describing the trained deep neural network;
A quantization step for quantizing the weight matrix reduced in the parameter reduction step; And
And an entropy coding step of scanning the quantized weight matrix in a predetermined direction in the quantization step, sequentially entropy coding the scanned weights, and outputting a bitstream,
In the parameter reduction step, a method of compressing a trained deep neural network, characterized in that at least one of a pruning technique and a low rank approximation technique is used.

Images