JP2021168042A - Information processing device, information processing method, and program - Google Patents

Abstract

To efficiently compress a learned model so as to contribute to acceleration of an arithmetic operation.SOLUTION: An information processing device is provided with an input/output arithmetic unit 14, a compression weight matrix calculation unit 15, and a re-learning unit 16. The input/output arithmetic unit 14 performs an arithmetic operation for inference data with a learned model and takes out input data and output data for performing a matrix operation in a specific layer being a compression object of the arithmetic operation. The compression weight calculation unit 15 performs an arithmetic operation for the input data taken out by the arithmetic unit 14 in a compression weight matrix in which a zero/nonzero pattern with an element at a specific suffix in the matrix being zero is applied to a matrix in a specific layer, and obtains a compression weight matrix with the weight being optimized by performing an arithmetic operation for reducing an error between the output data of the arithmetic operation result and the output data taken out by the arithmetic unit 14. The re-learning unit 16 re-learns the learned model in which the compression weight matrix obtained by the compression weight calculation unit 15 is applied to a specific layer by using correct data while maintaining a position of zero at zero.SELECTED DRAWING: Figure 4

Description

The present invention relates to an information processing device and an information processing method for performing an operation on a neural network used for artificial intelligence, and a program for executing the information processing method, and more particularly to a technique for reducing the amount of calculation when performing an operation on a neural network.

In general, in neural networks, deep neural networks (hereinafter referred to as "DNN") and convolutional neural networks (hereinafter referred to as "CNN"), which have particularly high recognition performance and prediction performance, are means such as Internet services, via the cloud, and on-board equipment. Is provided as an application to smartphones, automobile equipment, home appliances, factory equipment, robots, and the like.

WO2019 / 092900

Neural networks such as DNN and CNN, which are often used to realize conventional artificial intelligence functions, have a large amount of calculation, and a large-scale server is prepared for computer resources, or a graphic processing unit (hereinafter referred to as "GPU"). It is necessary to install additional units such as. For this reason, there is a problem that the introduction of artificial intelligence equipment and the mounting on the equipment become expensive, and a large amount of power consumption is required.

Therefore, it has been proposed to reduce the amount of calculation in each layer of the trained model constituting the neural network when performing the calculation of the neural network. For example, Patent Document 1 describes a technique for reducing the amount of calculation by deleting a part of the elements of the matrix when performing the matrix operation in each layer of the trained model constituting the neural network.

Deleting some elements of a matrix is equivalent to performing a matrix operation with the value to be deleted set to zero. By setting a part of the elements of the matrix to zero, the calculation including the zeroed part becomes unnecessary, the amount of calculation can be reduced, and the memory capacity required for the calculation can also be reduced.

As described above, as one of the conventional compression processes, it is known that the elements of the matrix having a small absolute value are set to zero and compressed.
In this compression technique, for example, as shown on the left side of FIG. 16, in a weight matrix of 3 rows × 3 columns, the value of the weight matrix is less than 1 (here, 0.1 or 0.2). ) Is converted into a compressed weight line example in which "0" is replaced as shown on the right side of FIG.

However, if the compression process is performed based on such a weight value, the position of the deletion target depends on the trained model, which may not contribute to speeding up the calculation.
In order to increase the calculation speed, it is preferable that the deletion position where the value is set to zero is a position that contributes to the speeding up of the operation. However, speeding up the operation of the neural network while maintaining the inference accuracy of the trained model has not been performed conventionally.

Further, conventionally, a method for determining the compression ratio of each layer, which suppresses the deterioration of the accuracy of the entire model, has not been known. Therefore, whether or not the compression ratio of each layer suppresses the deterioration of the accuracy of the entire model could not be known unless a process involving re-learning was performed and the compression model was actually created. Therefore, it has been difficult to efficiently compress the trained model while suppressing deterioration in accuracy.

An object of the present invention is to provide an information processing device, an information processing method, and a program capable of efficiently compressing a trained model so as to contribute to speeding up the calculation.

The information processing device of the present invention is an information processing device that compresses a trained model when performing a neural network calculation by applying a trained model composed of a plurality of layers to the reasoning data to be inferred. Therefore, it is provided with an input / output calculation unit that performs calculation of inference data by the trained model and extracts input data and output data when performing matrix calculation on a specific layer to be compressed at the time of the calculation.
Further, the information processing apparatus of the present invention is a compression weight matrix in which an element in a specific subscript of a matrix is set to zero and a zero / non-zero pattern in which other elements are the original values is applied to a matrix of a specific layer. A compression weight matrix with optimized weights is obtained by performing an operation to reduce the error between the output data of the calculation result obtained by performing the calculation of the extracted input data and the output data extracted by the input / output calculation unit. Re-learning the trained model in which the matrix calculation unit and the weight matrix obtained by the compression weight matrix calculation unit are applied to a specific layer, using the correct answer data while keeping the zero position of the compression weight matrix at zero. It has a learning department.

Further, the information processing method of the present invention is an information processing method that compresses a trained model when performing a neural network calculation by applying a trained model composed of a plurality of layers to the inference data to be inferred. The calculation procedure for extracting the input data and output data when the inference data is calculated by the trained model and the matrix operation is performed on the specific layer to be compressed at the time of the calculation, and the specific subscript of the matrix. The input data extracted by the calculation procedure is calculated by applying the zero / non-zero pattern, in which the elements in are zero and the other values are the original values, to the matrix of a specific layer, and the output data of the calculation result is performed. And the compression weight matrix calculation procedure to obtain the compression weight matrix with the appropriate weights by performing the operation to minimize the error with the output data extracted in the calculation procedure, and the weight matrix obtained by the compression weight matrix calculation procedure. It includes a retraining procedure in which a trained model applied to a particular layer is retrained using correct data while keeping the zero position of the compression weight matrix at zero.

Further, in the program of the present invention, each processing procedure of the above-mentioned information processing method is stepped and executed by a computer.

According to the present invention, each layer can be compressed with a pattern suitable for reducing the amount of calculation, and the trained model can be efficiently compressed.

It is a figure which shows the example of the layer structure of the trained model. It is a figure which shows the example of the convolution operation using the weight matrix of a trained model. It is a figure which shows the example which applied the zero pattern to a part of the weight matrix of FIG. It is a block diagram which shows the structural example of the information processing apparatus according to 1st Embodiment of this invention. It is a figure which shows the example of the re-learning part by the example of 1st Embodiment of this invention. It is a block diagram which shows the hardware configuration example of the information processing apparatus by the 1st Embodiment example of this invention. It is a flowchart which shows the flow of the processing procedure by the example of 1st Embodiment of this invention. It is a block diagram which shows the structural example of the information processing apparatus according to the 2nd Embodiment of this invention. It is a characteristic diagram which shows the example of the fluctuation of the correct answer rate by compression of each layer in the example of the 2nd Embodiment of this invention. It is a flowchart which shows the flow of the processing procedure by the 2nd Embodiment of this invention. It is a characteristic diagram which shows the example which compared the case of learning with the weight set to zero, and the case of re-learning after linear regression. It is a figure which shows the specific example of the model when this invention is applied to image recognition. It is a figure which shows the example which compressed the model shown in FIG. It is a figure which shows the example which relearned a specific layer of a model. It is a figure which shows an example of the data used in the process performed by the weight matrix selection part by this invention. It is a figure which shows the example which compressed the conventional weight matrix.

Hereinafter, examples of embodiments of the present invention will be described.
INDUSTRIAL APPLICABILITY The present invention applies a trained model composed of a plurality of layers to inference data to be inferred, and performs a process of compressing the trained model when performing a neural network operation.

[Trained model layer structure and weight matrix compression]
First, an example of a trained model that performs an operation on a neural network to which the present invention is applied will be described with reference to FIGS. 1 to 3.
FIG. 1 shows an example of the layer structure of the trained model.
As shown in FIG. 1, the trained model has a structure having a first layer L1, a second layer L2, a third layer L3, ..., An nth layer Ln (n is an arbitrary integer) and a plurality of layers. The calculation is performed on each of the layers L1 to Ln.

The calculation in each layer L1 to Ln can be divided into a linear matrix calculation and a calculation by a non-linear activation function, for example, as shown in an enlarged view of the third layer L3 in FIG. However, not all layers need to have both linear matrix calculations and non-linear activation function calculations.

In the case of a layer having a linear matrix calculation and an operation by a non-linear activation function, when the input to the target layer is x, first, in the linear matrix calculation, the linear function f (x) is linear. Matrix calculation is performed to obtain the value y. Further, the value y is converted into the value z by the operation by the non-linear activation function, and the value z becomes the output of the target layer.
Such operations are sequentially executed in each of the layers L1 to Ln, and an inference result is obtained as the output of the nth layer Ln in the final stage.

Here, for example, when the third layer L3 is compressed, the linear matrix calculation f (x) is converted into the compressed linear matrix calculation fc (x).
If the linear matrix calculation f (x) and the compressed linear matrix calculation fc (x) have the same input / output relationship, the inference accuracy will not be reduced by the compression.

FIG. 2 shows an example of the matrix representation of the linear matrix calculation f (x).
Here, as shown in the upper left of FIG. 2, when the convolution calculation of iw = 18, ih = 18, ic = 10, and stride = 1 is performed as the input x, as shown in the lower left of FIG. 2, kw * kh * it is expanded to ic = n matrix of _(matrix starting from _{x 11).} Here, let m be the number developed by stride.

As for the weight matrix, as shown in the upper center of FIG. 2, kw = 3, kh = 3, and ic = 10 are prepared for the number of filters (filter_num).
This weight matrix is represented by a matrix of oc = (filter_num) = l (a matrix _{starting from w 11} ) as shown in the lower center of FIG.
As a result, the output y obtained by multiplying the input x by the weight matrix is obtained as shown in the upper right of FIG. 2, and as shown in the lower right of FIG. 2, the matrix of _{the output y 11} (the matrix starting from _{y 11).} ).

FIG. 3 shows an example in which the linear matrix calculation f (x) shown in FIG. 2 is performed by a compressed linear matrix calculation.
In the example of FIG. 3, as shown in the lower part of the center of FIG. 3, replaced by a partial portion is zero (0) of the pattern of the weight matrix (w 'matrix starting from _11), portions of the non-zero, FIG. It is the same as the value of the weight matrix shown in the lower part of the center of 2. In the following description, the zero and non-zero patterns of this weight matrix are referred to as "zero / non-zero patterns".
The weight matrix shown in the lower center of FIG. 3 is multiplied by the input matrix to obtain the matrix of _{output y ′ 11} shown in the lower right of FIG. 3 (matrix starting from _{y ′ 11).}

The matrix operation for obtaining the output y ′ ₁₁ is an operation for a weight matrix having a zero / non-zero pattern, and multiplication and addition of zeros can be omitted. Therefore, the output y 'the number of operations for obtaining _11, it is possible to reduce than the number of operations for obtaining y ₁₁ shown in FIG. 2, the output y' is a matrix operation to obtain _11, the position of zero If it is a position that contributes to speeding up, it can be said that it contributes to reducing the amount of calculation of the neural network.

Here, comparing the uncompressed output y and the compressed output y', the result using the k-th column (= k-th filter) of the weight matrix and the i-th row of the input x is shown in Equation 1. It becomes a linear expression. w is the value of the weight matrix.

Here, if there are sufficient samples of the input x and the output y, it is possible to obtain _{the value w'jk of the compressed weight matrix by the method of least squares.}
Least squares method, 'the _(ik ² sum to a minimum, the value w of the weight of the compressed weight matrix error Σ _i ^m y ik _-y)' for each input data calculated from a plurality of input data _jk This is the method to be sought.
That is, using the input data and output data of the uncompressed matrix for the corresponding layer and the input data and output data of the compressed weight matrix to which the zero / non-zero pattern is applied, the error is calculated by the minimum square method. Processing is performed to minimize and obtain an appropriate value for the weight of the non-zero part of the compression weight matrix.
However, as described in the example of the embodiment described later, the method of applying the least squares method is an example, and other operations for reducing errors are performed to obtain a compressed weight matrix with appropriate weights. May be good. In the case of the least squares method, the error is minimized, but depending on the method to be applied, the error may not always be minimized as a result.
In each embodiment of the present invention described below, the least squares method or the like is applied to one or a plurality of layers to obtain _{the value w'jk of the compressed weight matrix that minimizes the sum of the errors.} , As a compressed matrix, the amount of calculation of the matrix of the trained model is reduced.

[Example of the first embodiment]
Next, in the first embodiment of the present invention, the process of compressing the trained model will be described with reference to FIGS. 4 to 7.
FIG. 4 is a block diagram showing a configuration example of the information processing device 10 of the embodiment.
The information processing device 10 of the embodiment of the present embodiment includes an inference data input unit 11, a trained model input unit 12, a correct answer data input unit 13, an input / output calculation unit 14, a compression weight matrix calculation unit 15, a relearning unit 16, and a re-learning unit 16. A learning model output unit 17 and a compression rate designation unit 18 are provided.
The inference data input unit 11 outputs inference data for performing inference processing. The trained model input unit 12 outputs the trained model learned in the past. Further, the correct answer data input unit 13 outputs correct answer data to the inference data output from the inference data input unit 11.

The input / output calculation unit 14 applies the trained model given by the trained model input unit 12 to the inference data input from the inference data input unit 11 to execute the inference process. The trained model at this time is composed of a plurality of layers as described with reference to FIG. In each layer, a linear matrix operation and a non-linear activation function operation are performed.

The correct answer data obtained in the correct answer data input unit 13 is data indicating the correct answer of the inference data input from the inference data input unit 11. For example, when the trained model given from the trained model input unit 12 is used for analysis processing to analyze the image data and determine an object included in the image, the image data is given to the input / output calculation unit 14 as inference data. .. The correct answer data obtained in the correct answer data input unit 13 is data indicating the correct answer of the object included in the image as the inference data at this time.

The inference data used for the calculation in the input / output calculation unit 14 is preferably the data used at the time of learning the trained model given from the trained model input unit 12, but is of the same type as the data at the time of learning. If it is data, it does not have to be the data used at the time of learning. For example, in the trained model, a picture of a dog taken in Japan may be used for learning, or data of a dog taken in the United States may be used as inference data.

The input / output calculation unit 14 can take out input data and output data of each layer (layer to be compressed) when performing inference calculation processing on inference data by applying the trained model.
The input data and output data of one or more layers to be compressed taken out by the input / output calculation unit 14 are supplied to the compression weight matrix calculation unit 15.

The compression weight matrix calculation unit 15 obtains output data by inputting the input data and output data of the compression target layer and one or more zero / non-zero patterns calculated by the input / output calculation unit 14. The zero / non-zero pattern is prepared in advance in the compression rate designation unit 18, and is supplied from the compression rate designation unit 18 to the compression weight matrix calculation unit 15. When the compression rate is specified by a user operation or the like, the compression rate specifying unit 18 selects one or more zero / non-zero patterns suitable for the compression rate, and selects one or more zero / non-zero patterns. Is supplied to the compression weight matrix calculation unit 15. The compression rate designation in the compression rate designation unit 18 may be performed automatically by the user operation.
The compression weight matrix calculation unit 15 gives input data to the compression weight matrix in which the input zero / non-zero pattern is applied to the weight matrix, and obtains output data of the compression weight matrix.
As already mentioned, in the zero / non-zero pattern, the value of the weighting coefficient at a specific plurality of points in the weighting matrix is set to zero, and the value of the weighting coefficient at other points is set by an operation such as the least squares method. It is a pattern to be the obtained value (non-zero).

As the zero / non-zero pattern, gamma = 2, gamma = 4, gamma = 8, gamma = 16 are prepared depending on the number of zeros in the matrix. In the zero / non-zero pattern of gamma = 2, the number of (non-zero) / (zero + non-zero) is 1/2. Similarly, for gamma = 4, gamma = 8, and gamma = 16, (non-zero) / (zero + non-zero) are 1/4, 1/8, and 1/16, respectively.
For example, in the zero / non-zero pattern of gamma = 4, 1/4 of the weight matrix is a non-zero coefficient value, and the remaining 3/4 is zero. Therefore, gamma = 16 is a zero / non-zero pattern having the highest compression ratio.

Next, a solution method for obtaining the optimum weighting coefficient value by the least squares method will be described.
For example, it is assumed that a plurality of input data x (x ₁ , ..., X _n ) and output data Y (Y ₁ , ..., Y _{n) are obtained.} Here, let x _i = (x _i1 , ···, x _id ) ∈ R ^d . At this time, the following multidimensional linear regression model is assumed with the parameters θ = (θ ₁ , ···, θ _d). That is, the parameter θ corresponds to one filter in the weight matrix. When obtaining the parameter, the element from θ to zero can be omitted in advance, and the column in which 0 is multiplied by 0 from the elements of θ can be omitted from x as well.
Y _i = x _i1 θ ₁ + x _i2 θ ₂ + ... x _id θ _d + εi, i = 1, ..., n
When this multidimensional linear regression model is expressed using a matrix, it is shown in Equation 2.

Therefore, Y = x _θ + ε.
The estimator by the least squares method can be obtained from the extreme value condition of the parameter that minimizes the square error shown in Equation 3.

When the extremum condition is calculated by differentiating the squared error with the parameter θ _{t, the equation 4 is obtained.}

When this equation is expressed using a matrix of input X, output Y, and parameter θ, it becomes equation 5.

Therefore, the solution of the least squares method is as shown in Equation 6.

Here, the optimum weight of the compression weight matrix is obtained by the least squares method, but the global solution may be obtained by using another method.
The least squares method attempts to solve a linear / non-linear classification / regression problem by minimizing the root-mean square error of the output. As a method other than the least squares method, the error can be minimized by a formulation that minimizes the absolute error of the output and a formulation that minimizes the cross entropy error (in the case of a classification problem).
That is, in the example of the present embodiment, applying the least squares method to obtain the optimum value of the non-zero weight of the compression weight matrix applies the least squares method to the scale for measuring the output error, and the square error. Is equivalent to using. However, another method may be applied instead of the scale for measuring the square error.

When the regression problem is linear (linear regression problem), there are mainly two methods for solving the problem of minimizing the square error (least squares method): a closed form method and various gradient descent methods. There is a unique solution to this problem due to the convexity of the objective function. In particular, the closed form solution and the solutions obtained by various gradient descent methods are in agreement. The various gradient descent methods referred to here include the batch gradient descent method (a method in which the entire training data is used at once for gradient calculation) and the stochastic gradient descent method (for each sample of training data). There are three methods to calculate the gradient) and the mini-batch gradient descent method that belongs to the middle of them.

The closed form method and various gradient descent methods described above have their own strengths and weaknesses. The calculation of the closed form solution has the advantage that it does not require the search for hyperparameters (parameters of the learning algorithm), unlike various gradient descent methods. On the other hand, the mini-batch gradient descent method and the stochastic gradient descent method are different from the closed form solution method in that they can easily handle data with a large number of samples and features. Furthermore, it is particularly important that various gradient descent methods can be applied to classification / regression problems that are not always linear. The classification / regression problem in machine learning (especially deep learning) is generally a non-linear problem with a large number of samples and features, and a (local) solution is usually obtained by the mini-batch gradient descent method.

Therefore, if the stochastic gradient descent method is used, even if there is a nonlinear calculation unit, the number of operations is small and the operation is close to a linear operation, so that the same method as the least squares method can be used. For example, the activation functions relu and relu6 can be mentioned.

Here, an example of finding a global solution by the stochastic gradient descent method will be described. In general, there are various gradient descent approaches to iterative solutions to linear regression problems. This approach implicitly solves the following equation, which is an equation equivalent to the root-mean-squared error minimization problem.

On the other hand, the iterative solution by the gradient descent method is an approach that solves like the equation 9 which is an iterative modification and converges to a global solution. This is also called the batch gradient descent method.

The stochastic gradient descent method is a method of replacing the entire data matrix X with a part of randomly sampled data.
The advantages of the stochastic gradient descent method are that it is easy to handle when both the number of samples and the number of features are large, and that it can be applied to nonlinear regression problems. However, the stochastic gradient descent method requires the setting of some hyperparameters such as learning rate, number of iterations, and batch size.

In this way, the compression weight matrix calculation unit 15 obtains the optimum weight coefficient value for the compression weight matrix compressed by applying the zero / non-zero pattern.
For example, as shown on the left side of FIG. 5, consider a configuration having layers L1, layer L2, layer L3, and layer L4 as trained models. When all of the four layers L1 to L4 are layers to be compressed, the compression weight matrix calculation unit 15 applies compression weight matrices W1'to W4 to which a zero / non-zero pattern is applied to each of the layers L1 to L4. ′ Is obtained. At this time, the values of the weighting coefficients of the compression weight matrices W1'to W4' are optimized by the least squares method or the like.

The data of the compressed compression weight matrices W1'to W4' obtained by the compression weight matrix calculation unit 15 is sent to the re-learning unit 16.
The re-learning unit 16 executes inference arithmetic processing on the inference data from the inference data input unit 11 with the model based on the compression weight matrices W1'to W4' as the initial value. Then, the re-learning unit 16 compares the inference result in the model based on the compression weight matrices W1'to W4' with the correct answer data given by the correct answer data input unit 13. Then, from this comparison result, as shown on the right side of FIG. 5, the re-learning unit 16 relearns the model including the compression weight matrices W1'to W4', and the re-learning model output unit 17 receives the re-learning model. Supply.
When the re-learning unit 16 re-learns, the compression weight matrix to which the zero / non-zero pattern is applied is re-learned while keeping the zero position at zero, and the compressed state is re-learned. It is designed to be maintained even after the event.

By re-learning in the re-learning unit 16 in this way, a re-learning model learned so as to reduce the error of the model as a whole is obtained from the re-learning model output unit 17. That is, since the weight matrices L1'to L4' obtained by the compression weight matrix calculation unit 15 minimize the error only with respect to the linear portion of the target layer, the error of the non-linear portion of each layer and the error , The error of the model as a whole is not small. Therefore, re-learning in the re-learning unit 16 is required, and an optimum re-learning model can be obtained from the re-learning unit 16.

FIG. 6 shows an example of hardware configuration when the information processing device 10 shown in FIG. 4 is configured by a computer.
The computer as the information processing device 10 includes a CPU (Central Processing Unit) 10a, a ROM (Read Only Memory) 10b, and a RAM (Random Access Memory) 10c, which are connected to the bus, respectively. Further, the information processing device 10 includes a non-volatile storage 10d, a network interface 10e, an input unit 10f, and a display unit 10g.

The CPU 10a is an arithmetic processing unit that reads out from the ROM 10b the program code of the software that executes the arithmetic processing in the input / output arithmetic unit 14, the compression weight matrix calculation unit 15, and the relearning unit 16, and executes the arithmetic processing. Variables, parameters, etc. generated during the arithmetic processing are temporarily written in the RAM 10c.

For the non-volatile storage 10d, for example, a large-capacity information storage unit such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) is used. Data such as correct answer data, inference data, and trained model are stored in the non-volatile storage 10d.

For the network interface 10e, for example, a NIC (Network Interface Card) or the like is used.
The input unit 10f receives an input operation of a user who operates the information processing device 10. For example, the input unit 10f accepts an input operation such as which layer of the trained model is to be compressed. However, the layer to be compressed may be automatically selected by the calculation in the information processing apparatus 10.
The calculation result of the information processing apparatus 10 is displayed on the display unit 20g.

The information processing device 10 is configured by the computer shown in FIG. 6 as an example, and may be configured by a device other than the computer that performs arithmetic processing. For example, a part or all of the functions performed by the information processing apparatus 10 may be realized by hardware such as FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit).
Another example is that the information processing device 10 is configured to include an input unit 10f and a display unit 10g, and the information processing device 10 is a computer that does not have either one or both of the input unit 10f and the display unit 10g. It may be configured as.

FIG. 7 is a flowchart showing a flow of processing for obtaining a compressed relearning model by the information processing apparatus 10.
First, the inference data, the trained model, and the correct answer data are prepared in the inference data input unit 11, the trained model input unit 12, and the correct answer data input unit 13, respectively (step S11). Then, the input / output calculation unit 14 executes the calculation processing of each layer including the compression target layer using the trained model (step S12).

After that, the input / output calculation unit 14 takes out the input data and the output data of the compression target layer from the calculated trained model and supplies them to the compression weight matrix calculation unit 15 (step S13). Then, when the compression target layer is all the layers of the trained model, the input / output calculation unit 14 supplies the input data and the output data of all the layers to the compression weight matrix calculation unit 15. Which layer is the compression target layer is set by, for example, a user operation.

The compression weight matrix calculation unit 15 applies one or a plurality of zero / non-zero patterns prepared in advance to the linear weight matrix of the layer to be compressed to obtain a compression weight matrix, and the compression weight matrix is non-zero. A compressed weight matrix in which the value of the weighting coefficient is set to an appropriate value by the minimum square method is obtained (step S14). The zero / non-zero pattern to be applied is determined by the designation of the compression rate in the compression rate designation unit 18.

After the compressed weight matrix calculation unit 15 obtains the compressed weight matrix of the layer to be compressed, the re-learning unit 16 relearns the entire model including the corresponding compressed weight matrix to obtain a re-learning model ( Step S15). However, when the re-learning model is obtained, the zero position of the compression weight matrix is maintained at zero so that the compressed state is maintained.

The compression weight matrix used by the relearning unit 16 for relearning does not have to be the compression weight matrix calculated from the same trained model.
For example, the re-learning unit 16 may repeat the following loop.
1. 1. Let the model to be compressed be a trained uncompressed model.
2. The input data x and the output data y of the i-th layer are calculated.
3. 3. Compute the compression weight matrix for the i-th layer.
4. Relearn the weights of the i-th layer as a compressed weight matrix.
5. 4. The model to be compressed. Let it be the weight relearned in.
6. As i = i + 1, 1. From 5. Repeat the process of.

As described above, according to the embodiment of the present embodiment, the least squares method or the like is applied to one or a plurality of layers to obtain a compressed weight matrix that minimizes the error. Then, the model can be retrained using the compressed matrix to obtain a trained model in which the amount of arithmetic processing is appropriately compressed.

[Example of the second embodiment]
Next, in the second embodiment of the present invention, the process of compressing the trained model will be described with reference to FIGS. 8 to 9.
FIG. 8 is a block diagram showing a configuration example of the information processing device 20 of the embodiment.
The information processing device 20 of the embodiment of the present embodiment includes an inference data input unit 21, a trained model input unit 22, a correct answer data input unit 23, an input / output calculation unit 24, a compression weight matrix calculation unit 25, and a weight matrix selection unit 26. It includes a re-learning unit 27, a re-learning model output unit 28, and a compression rate designation unit 29.

The input / output calculation unit 24 and the compression weight matrix calculation unit 25 perform the same processing as the input / output calculation unit 14 and the compression weight matrix calculation unit 15 described in the first embodiment. The compression rate designation unit 29 is also subjected to the same processing as the compression rate designation unit 18 described in the first embodiment. However, in the case of the present embodiment, a plurality of zero / non-zero compression rates can be selected so that the result of applying the zero / non-zero patterns of a plurality of compression rates in each layer in the weight matrix selection unit 26 described later can be selected. It is preferable to select a non-zero pattern and supply it to the compression weight matrix calculation unit 25.
Further, the inference data, the trained model, and the correct answer data input from the inference data input unit 21, the trained model input unit 22, and the correct answer data input unit 23 are also the inference data input unit 11 shown in FIG. It is the same as the inference data, the trained model, and the correct answer data input from the model input unit 12 and the correct answer data input unit 13.

Then, in the first embodiment, the point that the output data obtained by the compression weight matrix calculation unit 25 using the plurality of zero / non-zero patterns is supplied to the weight matrix selection unit 26 is the first embodiment. Is different. Further, in connection with this, the weight matrix selection unit 26 is supplied with the correct answer data from the correct answer data input unit 23, and the weight matrix selection unit 26 is supplied with the target calculated by the input / output calculation unit 24. The output data of the calculation result of the matrix before compression of the layer is supplied. However, if the weight matrix selection unit 26 does not require the correct answer data, the correct answer data input unit 23 does not supply the correct answer data.

The weight matrix selection unit 26 determines whether the compression weight matrix is good or bad calculated by the compression weight matrix calculation unit 25, and selects the optimum compression weight matrix. For example, the weight matrix selection unit 26 acquires output data obtained by using a plurality of zero / non-zero patterns having different compression rates, compares it with the output data of the same layer before compression, and determines the correct answer rate in advance. Select a compression weight matrix that uses a zero / non-zero pattern that is above the threshold and has the highest compression ratio.

FIG. 9 shows an example in which the correct answer rate for each layer varies depending on the compression rate. The horizontal axis of FIG. 9 shows the layers, and the vertical axis shows the accuracy rate of the model when each layer is compressed. Here, as for the value of the correct answer rate, the closer it is to 1, the higher the correct answer rate is, and the closer it is to 0, the lower the correct answer rate is.
The data G0 shown in FIG. 9 is the correct answer rate of the trained model when each layer is not compressed. In the example of FIG. 9, the uncompressed data G0 has a correct answer rate of about 0.7 in each layer.

Further, the data G2, G4, G8, and G16 shown in FIG. 9 have a compression ratio of gamma = 2, gamma = 4, gamma = 8, gamma = 16, that is, (non-zero) / (zero + non-zero) of 1. The correct answer rates at / 2, 1/4, 1/8, and 1/16 are shown.
For example, when the correct answer rate is 0.6 as the threshold value, consider which compression rate of G0, G2, G4, G8, and G16 should be selected.
The weight matrix selection unit 26 selects a weight having a compression ratio of G2, that is, gamma = 2 in the first layer shown at the left end.

Further, in the second layer, a compression weight matrix using a zero / non-zero pattern of gamma = 8 shown as data G8 having the highest compression rate among the accuracy rate exceeding 0.6 is selected.
In this way, the weight matrix selection unit 26 selects the compression weight matrix using the zero / non-zero pattern having the highest compression rate among the thresholds exceeding 0.6. Then, by making such a selection for all layers, a model including a compression weight matrix is generated.

Here, an example has been described in which the weight matrix selection unit 26 uses the correct answer rate to select a compression weight matrix using a zero / non-zero pattern having a high compression rate. However, the degree of approximation is not limited to the accuracy rate of the model, and other indexes representing the degree of approximation, which will be described later, may be used.
Further, the weight matrix selection unit 26 selects, for example, a compression weight matrix having a good degree of approximation if the ratio of zero and non-zero is the same compression weight matrix.

Here, the degree of approximation is obtained, for example, as follows. First, the input data is input to the compression weight matrix calculation unit 25, and an error between the value output from the compression weight matrix calculation unit 25 and the output data when not compressed is obtained. The following can also be used as an index indicating the degree of approximation. That is, the coefficient of determination, the correlation coefficient, and the compression weight matrix can be inserted into the model, and the error from the correct answer data when the inference data is input can be adopted as the degree of approximation.

Further, when comparing the compression weight matrices of a plurality of layers, if the same degree of reduction in operations is expected, a compression weight matrix with good accuracy is selected. It is preferable that the weight matrix selection unit 26 determines each of these items and selects the compression weight matrix that seems to be the best.
When selecting a compression weight matrix that seems to be excellent, a user instruction or the like may be used to determine which item should be prioritized. For example, the degree of approximation may be prioritized for selection, or the deletion of the amount of calculation may be prioritized for selection, depending on the situation.

The model data generated by the weight matrix selection unit 26 is sent to the relearning unit 27. The re-learning unit 27 uses the obtained model as an initial value and executes inference arithmetic processing on the inference data from the inference data input unit 21. Then, the re-learning unit 27 re-learns the model including the compressed weight matrix by comparing the inference result in the model including the compressed weight matrix with the correct answer data given from the correct answer data input unit 23. It learns and supplies the re-learning model to the re-learning model output unit 28. Even when obtaining the re-learning model here, the zero position of the compression weight matrix is assumed to be maintained at zero so that the compressed state is maintained.

FIG. 10 is a flowchart showing a flow of processing for obtaining a compressed relearning model by the information processing apparatus 20.
First, the inference data input unit 21, the trained model input unit 22, and the correct answer data input unit 23 prepare the inference data, the trained model, and the correct answer data, respectively (step S21). Then, the input / output calculation unit 24 executes the calculation processing of each layer including the compression target layer using the trained model (step S22).

After that, the input / output calculation unit 24 takes out the input data and the output data of the compression target layer from the calculated trained model and supplies them to the compression weight matrix calculation unit 25 (step S23). When the compression target layer is all the layers of the trained model, the input / output calculation unit 24 supplies the input data and the output data of all the layers to the compression weight matrix calculation unit 25. Which layer is the compression target layer is set by, for example, a user operation.

The compression weight matrix calculation unit 25 obtains a compression weight matrix to which one or a plurality of zero / non-zero patterns prepared in advance are applied to the linear weight matrix of the layer to be compressed, and then the non-zero compression weight matrix. With respect to the coefficient value of the weight of, an appropriate value is obtained by the minimum square method or the like (step S24).
Then, the output data of the obtained compressed weight matrix is supplied to the weight matrix selection unit 26.
The weight matrix selection unit 26 compares the output data of the obtained calculation result with the output data of the corresponding layer of the input / output calculation unit 24, and compares the degree of approximation of both output data and the applied zero / non-zero pattern. An appropriate compression weight matrix that matches the given conditions is selected from the ratio of zeros, the amount of calculation reduction, and the like (step S25).

When the linear weight matrix of the layer to be compressed is selected by the weight matrix selection unit 26, the relearning unit 27 relearns the entire model including the selected linear weight matrix to obtain a relearning model (step S26). .. When obtaining the re-learning model, the zero position of the compression weight matrix is assumed to be maintained at zero so that the compressed state is maintained.

As described above, even when the weight matrix selection unit 26 performs the process of selecting the linear weight matrix of the layer to be compressed, the amount of arithmetic processing is appropriately compressed as in the first embodiment. You can get the trained model that has been trained.

FIG. 11 shows a case where the zero position of the compression weight matrix is simply maintained at zero (without performing linear regression) and a case where the compression weight matrix is calculated using the least squares method and then relearned. It is a comparison of the accuracy in the case of.
The horizontal axis of FIG. 11 is the number of epochs (number of arithmetic processes) of the model, and the vertical axis is the accuracy.
The accuracy of inference in the trained model d1 when the compression weight matrix is calculated using the least square method and then relearned is more accurate than the zero position of the compression weight matrix, regardless of the number of epochs. Since it is higher than the trained model d2 acquired as the maintained state, it can be seen that the trained model in which the amount of arithmetic processing is appropriately compressed by the processing of the present invention can be acquired.

[Example of compressed concrete model]
Next, a specific example of compressing the trained model by applying the present invention will be described with reference to FIGS. 12 to 14.
First, as shown in FIG. 12, a trained model that recognizes 10 types of objects such as an airplane and a car is acquired from an image. When the trained model already prepared is applied and the image shown in the upper left is recognized, the inference results are 0.90 for an airplane, 0.01 for a car, ..., 0.01 for a horse. Suppose.

In the prepared trained model, there are layers such as an input layer (input_1), a two-dimensional convolution layer (conv_1, conv_2, conv_3), a tensor reshape layer (flatten_1), and a fully connected layer (dense_1). Weighting factors are set for the convolution layer and the fully connected layer.
The specific numerical values on the right side of FIG. 12 show an example of the value of the weight matrix in the two-dimensional convolution layer (conv_1).

FIG. 13 shows an example of data when the specific layer of the trained model shown in FIG. 12 has a compression ratio of gamma = 2, gamma = 4, gamma = 8, and gamma = 16. It shows that as the compression ratio increases, the number of zeros increases.

FIG. 14 shows an example in which, when a layer having a specific compression ratio of the trained model is selected, the layer is further modified by retraining. FIG. 14 shows a weight matrix of gamma = 8, but it is different from the weight matrix of gamma = 8 shown in FIG.

FIG. 15 is a diagram showing a specific example in which the weight matrix selection unit 26 selects the compression weight matrix according to the present invention in the second embodiment of the present invention.
FIG. 15 shows the total number of parameter reductions for the three layers (conv_1, conv_2, conv_3) when the uncompressed uncompressed layers and the compressed compressibility are combined, and an approximation of the average for each layer. The degree and the minimum value of the degree of approximation are shown in a list.
At the time of selection by the weight matrix selection unit 26, for example, when the total number of parameter reductions is equal to or greater than a predetermined value, the one with the highest average degree of approximation in each layer or the combination with the highest degree of approximation is selected. select.

Alternatively, both the average degree of approximation and the lowest degree of approximation may be used for selection. For example, if the total number of parameter reductions is greater than or equal to the specified value and the minimum value of the degree of approximation is 0.43, the minimum value is 0.05 (predetermined value) to 0.43. It is also possible to select the weight matrix with the highest degree of approximation of the average in each layer.
However, it is not necessary to create the data shown in FIG. 15 for all layers. For example, when the ratio of zeros in the layer having the lowest degree of approximation is further increased, the minimum value of the degree of approximation often decreases, so such a calculation can be omitted.

In the example of the embodiment of the present invention, a two-dimensional matrix has been described, but it is not necessary to be two-dimensional in practice, and it goes without saying that the present invention can be applied to a matrix of three or more dimensions.
Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and other application examples, as long as they do not deviate from the scope of the matters described in the claims. Needless to say, it includes a modified example.

10 ... Information processing device, 10a ... CPU, 10b ... RAM, 10c ... ROM, 10d ... Non-volatile storage, 10e ... Network interface, 10f ... Input unit, 10g ... Display unit, 11 ... Inference data input unit, 12 ... Learned Model input unit, 13 ... Correct data input unit, 14 ... Input / output calculation unit, 15 ... Compression weight matrix calculation unit, 16 ... Relearning unit, 17 ... Relearning model output unit, 18 ... Compression rate specification unit, 20 ... Information Processing device, 21 ... Inference data input unit, 22 ... Learned model input unit, 23 ... Correct answer data input unit, 24 ... Input / output calculation unit, 25 ... Compressed weight matrix calculation unit, 26 ... Weight matrix selection unit, 27 ... Re Learning unit, 28 ... Re-learning model output unit, 29 ... Compression rate specification unit

Claims (8)

An information processing device that compresses the trained model when performing neural network operations by applying a trained model composed of a plurality of layers to the inference data to be inferred.
An input / output calculation unit that performs an operation on the inference data using the trained model and extracts input data and output data when performing a matrix operation on a specific layer to be compressed at the time of the operation.
It is obtained by calculating the input data taken out by the input / output calculation unit with a compression weight matrix in which a zero / non-zero pattern in which the element in a specific subscript of the matrix is zero is applied to the matrix of the specific layer. A compression weight matrix calculation unit that obtains a compression weight matrix with appropriate weights by performing an operation to reduce the error between the output data of the calculation result and the output data extracted by the input / output calculation unit.
The trained model in which the compression weight matrix obtained by the compression weight matrix calculation unit is applied to the specific layer is retrained using correct answer data while keeping the zero position of the compression weight matrix at zero. An information processing device equipped with a re-learning unit. The information processing apparatus according to claim 1, wherein the operation for reducing the error performed by the compression weight matrix calculation unit is a process for obtaining a weight having the minimum error by the least squares method. The compression weight matrix calculation unit prepares one or more patterns as the zero / non-zero pattern to be used at the time of calculation.
The output data of the calculation results of the plurality of patterns obtained by the compression weight matrix calculation unit is compared with the output data extracted by the input / output calculation unit, and any one of the compression weight matrix or the uncompressed weight is compared. It also has a weight matrix selection section that selects a matrix.
The information processing apparatus according to claim 1, wherein the re-learning unit relearns the trained model in which the weight matrix selected by the weight matrix selection unit is applied to the specific layer. The plurality of patterns include a plurality of zero / non-zero patterns having different compression rates, which is a ratio of the number of zero elements and the number of non-zero elements included in the matrix.
The weight matrix selection unit is a compression that is a ratio of zero to an approximate state of the output data of the calculation result in the weight matrix to which the respective zero / non-zero patterns are applied and the output data extracted by the information processing operation unit. The information processing apparatus according to claim 3, wherein one of the compression weight matrices is selected by comprehensively determining the rate. It is an information processing method that compresses the trained model when performing a neural network operation by applying a trained model composed of a plurality of layers to the inference data to be inferred.
By performing the calculation of the inference data by the trained model, a calculation procedure for extracting input data and output data when performing a matrix operation on a specific layer to be compressed at the time of the calculation, and a calculation procedure.
The input data extracted in the above calculation procedure is calculated by applying the zero / non-zero pattern in which the element in the specific subscript of the matrix is zero to the matrix of the specific layer, and the calculation result is output. A compression weight matrix calculation procedure for obtaining a compression weight matrix with appropriate weights by performing an operation to reduce the error between the data and the output data extracted in the above calculation procedure.
The trained model in which the weight matrix obtained in the compression weight matrix calculation procedure is applied to the specific layer is retrained using correct answer data while keeping the zero position of the compression weight matrix at zero. Information processing methods, including learning procedures. One or more patterns are prepared as the zero / non-zero pattern to be used at the time of calculation in the compression weight matrix calculation procedure.
The output data of the calculation results of the plurality of patterns obtained in the compression weight matrix calculation procedure is compared with the output data extracted in the calculation procedure, and any one of the compression weight matrix or the uncompressed weight matrix is obtained. Including the weight matrix selection procedure to select
The information processing method according to claim 5, wherein the trained model in which the weight matrix selected in the weight matrix selection procedure is applied to the specific layer is relearned in the relearning procedure. A program that causes a computer to execute information processing that compresses the trained model when a trained model composed of a plurality of layers is applied to the inference data to be inferred and a neural network operation is performed.
A calculation step of performing the calculation of the inference data by the trained model and extracting input data and output data when performing a matrix operation on a specific layer to be compressed at the time of the calculation, and
The input data extracted in the calculation step is calculated by applying the zero / non-zero pattern in which the element in the specific subscript of the matrix is zero to the matrix of the specific layer, and the calculation result is output. A compression weight matrix calculation step for obtaining a compression weight matrix with appropriate weights by performing an operation to reduce the error between the data and the output data extracted in the calculation step.
Re-learning the trained model using the correct answer data by applying the weight matrix obtained in the compression weight matrix calculation step to the specific layer while keeping the zero position of the compression weight matrix at zero. Learning steps and
A program that causes the computer to execute. One or more patterns are prepared as the zero / non-zero pattern to be used at the time of calculation in the compression weight matrix calculation step.
As a step executed by the computer, the output data of the calculation results of the plurality of patterns obtained in the compression weight matrix calculation step is compared with the output data extracted in the calculation step, and any one of the compression weights is compared. It further includes a weight matrix selection step to select a matrix or an uncompressed weight matrix.
The program according to claim 7, wherein the trained model in which the weight matrix selected in the weight matrix selection step is applied to the specific layer is relearned in the relearning step.

Images