JP2021005242A - Information processing device, information processing program, and information processing method - Google Patents

Abstract

To accelerate convolution calculation.SOLUTION: An information processing device comprises: a calculation section 42 in which, among combinations of t and q in which a sum of each element from a plurality of first matrices g and a plurality of second matrices d of t×t does not exceed a count of data that can be stored in each of a q count from storage areas R#0 to R#7 of a register G#0, a combination in which a calculation time minimizes when each of calculation cores C#0 to C#3 in the q count executes convolution calculation for the plurality of first matrices g and the plurality of second matrices d parallelly in the Winograd algorithm, is calculated; and an output section which outputs a program 50 for causing a computation machine 10 to execute, processing for storing the first matrices g and the second matrices d of the t×t to each of the storage areas R#0 to R#3 in the q count, and processing in which each of the calculation cores C#0 to C#3 in the q count performs the convolution calculation for the first matrices g and the second matrices d using the Winograd algorithm.SELECTED DRAWING: Figure 22

Description

The present invention relates to an information processing device, an information processing program, and an information processing method.

Machine learning using a multi-layered neural network is called deep learning and is applied to various fields. Various calculations are performed in each layer of the deep learning. For example, in the convolution layer, a convolution calculation is performed between the image data and the filter, and the result is output in the subsequent stage. Since the convolution calculation is a matrix-to-matrix calculation, the amount of calculation is large, which contributes to a delay in the learning processing speed. Therefore, the Winograd algorithm has been proposed as an algorithm for reducing the amount of convolution calculation.

“Fast Algorithms for Convolutional Neural Networks”, Andrew Lavin et al., The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 4013-4021 “Deep Residual Learning for Image Recognition”, Kaiming He et al., The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770-778

However, the Winograd algorithm has room for improvement in that it further speeds up the processing of convolution calculations.

According to one aspect, the present invention aims to speed up the convolution calculation.

According to one aspect, the total number of elements of each of the plurality of first matrices and the plurality of second matrices of t rows and t columns is the data that can be stored in each of the q storage areas of the registers. Of the combinations of t and q that do not exceed the number of t and q, each of the q calculation cores corresponding to each of the q storage areas is a plurality of the first matrix and a plurality of the second matrix. A calculation unit that calculates the combination that minimizes the calculation time when the convolution calculation is executed in parallel by the Winograd algorithm, and a plurality of said firsts in each of the q storage areas using the calculated combination of t and q. The process of storing the matrix of 1 and the plurality of the second matrix of t rows and t columns, and each of the q calculation cores of the first matrix and the second matrix using the Winograd algorithm. An information processing apparatus is provided that includes an output unit that outputs a program for causing a computer having the calculation core and the register to execute a process of performing a convolution calculation.

According to one aspect, the convolution calculation can be speeded up.

FIG. 1 is a diagram schematically showing a flow of deep learning processing. FIG. 2 is a diagram schematically showing a convolution calculation performed in the convolution layer. 3 (a) to 3 (c) are diagrams schematically showing the convolution calculation of the bottom matrix and the weight matrix. 4 (a) to 4 (c) are diagrams schematically showing the Winograd algorithm in the forward processing. FIG. 5 is a hardware configuration diagram of a computer for performing convolution calculation in deep learning and the like. FIG. 6A is a hardware configuration diagram of one DPU-chain, and FIG. 6B is a hardware configuration diagram of one DPU. FIG. 7 is a hardware configuration diagram of each DPE. FIG. 8 is a hardware configuration diagram of DPE0. FIG. 9 is a diagram for explaining the line numbers assigned to each of the banks R # 0 to R # 7. 10 (a) to 10 (c) are schematic views (No. 1) for explaining the sequential method. 11 (a) to 11 (c) are schematic views (No. 2) for explaining the sequential method. FIG. 12 is a schematic diagram for explaining the multicast method. FIG. 13 is a diagram schematically showing the contents of the respective registers G # 0 of each DPE. FIG. 14 is a diagram schematically showing the array elements of the array g in the main memory. FIG. 15 is a diagram showing the contents of register G # 0 of DPE0 immediately after being transferred by the multicast method. FIG. 16 is a diagram showing the contents of register G # 0 of DPE0 after alignment. FIG. 17 is a diagram showing the contents of each register G # 0 of DPE0 to DPE7 after alignment. FIG. 18 is a schematic diagram of bank R # 0 of register G # 0 of DPE0. FIG. 19 is a hardware configuration diagram of the information processing device according to the present embodiment. FIG. 20 is a functional configuration diagram of the information processing device according to the present embodiment. FIG. 21 is a functional block diagram of the computer. FIG. 22 is a diagram showing the contents of each register G # 0 of DPE0 to DPE7 in which the arrays d and g are stored by the storage unit when the forward processing is performed in the present embodiment. 23 (a) and 23 (b) are diagrams (No. 1) showing the contents of each register G # 0 to G # 3 of DPE0 when the calculation unit performs convolution calculation by the Winograd algorithm in the present embodiment. FIG. 24 is a diagram (No. 2) showing the contents of each register G # 0 to G # 3 of DPE0 when the calculation unit performs convolution calculation by the Winograd algorithm in the present embodiment. FIG. 25 is a diagram (No. 3) showing the contents of each register G # 0 to G # 3 of DPE0 when the calculation unit performs convolution calculation by the Winograd algorithm in the present embodiment. FIG. 26 is a schematic diagram showing the calculation of the formula (19) of the present embodiment in step order. FIG. 27 is a schematic diagram showing the calculation of the formula (21) of the present embodiment in step order. FIG. 28 is a flowchart of the information processing method according to the present embodiment. 29 (a) to 29 (c) are schematic diagrams when the convolution calculation of the top matrix and the weight matrix is performed by the Winograd algorithm in the backword processing according to the present embodiment. FIG. 30 is a diagram showing the contents of each register G # 0 of DPE0 to DPE7 in which the arrays y and g are stored by the storage unit according to the present embodiment. 31 (a) and 31 (b) are schematic views when the convolution calculation of the top matrix and the bottom matrix is performed by the Winograd algorithm in the backword processing according to the present embodiment. 32 (a) to 32 (c) are schematic views when the convolution calculation of the top matrix and the bottom matrix is performed by the Winograd algorithm in the backword processing according to the present embodiment. FIG. 33 is a diagram showing the contents of each register G # 0 of DPE0 to DPE7 in which the arrays y and d are stored by the storage unit according to the present embodiment. FIG. 34 is a diagram showing the contents of the register G # 0 of DPE0 in which the arrays d and g are stored by the storage unit when 1 × 1 convolution is performed in the present embodiment. FIG. 35 is a diagram showing the contents of the register G # 0 of DPE0 in which the small bottom matrix d is stored by the storage unit according to the present embodiment at the time of batch normalization. 36 (a) and 36 (b) are diagrams showing the contents of register G # 0 of DPE0 for explaining the calculation performed by the calculation unit according to the present embodiment at the time of batch normalization.

Prior to the description of the present embodiment, the matters examined by the inventor of the present application will be described.

FIG. 1 is a diagram schematically showing a flow of deep learning processing.
In deep learning, the neural network is made to learn the characteristics of the identification target by performing supervised learning regarding the identification target such as an image. By using the neural network trained in this way, the identification target can be identified.

A neural network is a network in which units that imitate neurons in the brain are hierarchically connected. Each unit receives data from another unit and passes the data to the other unit. In a neural network, various identification targets can be identified by changing the parameters of the unit by learning.

In the following, a convolutional neural network (CNN) used for image recognition will be described with reference to FIG.

This neural network has a hierarchical structure with a convolution layer, a sub-sampling layer, and a fully-connected layer. In the example of FIG. 1, the convolution layer and the subsampling layer are alternately provided twice, but more layers may be provided. Further, a plurality of fully bonded layers may be provided. The hierarchical structure of the neural network and the configuration of each layer may be predetermined by the designer according to the object to be identified.

The process of identifying an image with a neural network is also called a forward process. In the forward process, as shown in FIG. 1, a plurality of convolution layers and pooling layers are alternately repeated from left to right. Finally, the fully connected layer identifies the identification target reflected in the image.

The process of learning an image with a neural network is also called backward process. In backward processing, the error between the identified result and the correct answer is obtained, and the error is propagated back to the neural network from right to left, and the parameters of each layer of the convolutional neural network are changed.

FIG. 2 is a diagram schematically showing a convolution calculation performed in the convolution layer.

In FIG. 2, a convolution calculation of a bottom matrix in which pixel data of an input image is stored in each element and a weight matrix representing a filter acting on the input image is illustrated. In this example, a plurality of bottom matrix and weight matrix are prepared, and convolution is performed between them.

Each of the plurality of bottom matrices is identified by the batch number N and the input channel number Cin. On the other hand, the weight matrix is identified by the output channel number Cout and the input channel number Cin.

In the example of FIG. 2, the convolution calculation is performed as follows.
First, select one combination of batch number N and output channel number Cout. For example, N = 0 and Cout = 0.

Then, among the combinations of the plurality of bottom matrices having the selected batch number N and the plurality of weight matrices having the selected output channel number Cout, the combination having the same input channel number Cin is selected. For example, when N = 0 and Cout = 0 as described above, a bottom matrix with N = 0 and Cin = 0 and a weight matrix with Cout = 0 and Cin = 0 are selected.

Then, a convolution is performed between these selected bottom and weight matrices. The matrix obtained by the convolution is hereinafter referred to as the top matrix.

256 top matrices can be obtained by performing such convolution in each bottom matrix and weight matrix of Cin = 0 to 255 with the batch number N and the output channel number Cout fixed. After that, by adding each of these 256 top matrices, one output matrix specified by the batch number N and the output channel number Cout is obtained.

Further, by performing such a calculation while changing the number of batches N and the output channel number Cout, an output matrix of the total number of batches N × the total number of output channel numbers Cout is finally obtained. In the example of FIG. 2, 64 × 384 output matrices are obtained.

In this way, the convolution calculation of the plurality of bottom matrices and the plurality of weight matrices is performed.

In such a convolution calculation, as described above, the convolution calculation is performed between two bottom matrices and a weight matrix having the same input channel number Cin. Therefore, the convolution calculation of these matrices will be described in detail.

3 (a) to 3 (c) are diagrams schematically showing the convolution calculation of the bottom matrix and the weight matrix.

First, as shown in FIG. 3A, a bottom matrix and a weight matrix to be convolved are prepared. In this example, the bottom matrix is a 13 × 13 square matrix and the weight matrix is a 3 × 3 square matrix.

Next, as shown in FIG. 3B, a 15 × 15 matrix M is obtained by 0 padding around the bottom matrix.

Subsequently, as shown in FIG. 3C, a minor matrix P _ij having the same size as the weight matrix is extracted in the matrix M. In the following, the elements of the submatrix P _ij with k rows and l columns are represented by (P _ij ) _kl (0 ≤ k, l ≤ 2), and the elements of the weight matrix k rows and l columns are g _kl (0 ≤ k, l). It is represented by ≤2).

The matrix obtained by convolving the matrix M and the weight matrix is called the top matrix as described above. In this case, each element r _ij of the top matrix can be calculated from the following equation (1).

However, in this method, in order to obtain one element r _ij of the top matrix, it is necessary to multiply by the same number as the number of elements (3 × 3) of the weight matrix, and it is not possible to realize high-speed convolution calculation.

The Winograd algorithm is known as an algorithm for accelerating the convolution calculation. Therefore, the Winograd algorithm will be described below.

As described above, deep learning includes forward processing and backward processing, but here, the Winograd algorithm in forward processing will be described.

4 (a) to 4 (c) are diagrams schematically showing the Winograd algorithm in the forward processing.

First, as shown in FIG. 4A, a small bottom matrix d of t × t is cut out from the bottom matrix. Note that t is a natural number.
Next, the small top matrix y is obtained according to the following equation (2).

小top行列yは、top行列の一部を形成する行列である。

The small top matrix y is a matrix that forms part of the top matrix.

Further, B, G, and A in the equation (2) are constant matrices. The elements and sizes of these constant matrices B, G, and A change according to the size of each of the matrices g and d. For example, when the size of the weight matrix g is 3 × 3 and the size of the small bottom matrix d is 4 × 4, the elements and sizes of the constant matrices B, G, and A are as shown in the following equation (3). Become.

The operator "◎" in Eq. (2) is a multiplication for each element of the matrix. For example, if each element of any matrix U and V of the same type is u _ij , v _ij , and the ij element of U ◎ V is (U ◎ V) _ij , then (U ◎ V) _ij = u _ij v _ij It becomes.

Next, as shown in FIG. 4 (b), the position where the small bottom matrix d is cut out from the bottom matrix is shifted by two columns from the case of FIG. 4 (a), and the same calculation as above is performed for the cut out small bottom matrix d. I do. The small top matrix y obtained in this way forms a block next to the small top matrix y obtained in FIG. 4A in the top matrix.

By shifting the positions for cutting out the small bottom matrix d from the bottom matrix by two in the column direction and the row direction in this way, as shown in FIG. 4C, a top matrix formed by each small top matrix y is obtained. be able to.

This completes the convolution calculation of the bottom matrix and top matrix using the Winograd algorithm.

In the Winograd algorithm of equation (2), once the matrix GgG ^T and the matrix B ^T dB are created, the convolution can be performed simply by calculating the product of each element, so the convolution calculation can be performed at high speed. It can be carried out.

The inventor of the present application estimated the calculation time when the size of the weight matrix g is 3 × 3 and the size of the small bottom matrix d is 4 × 4 as in this example. As a result, in the examples of FIGS. 3A to 3C in which the Winograd algorithm is not used, the calculation time is 1152 cycles. The "cycle" is equivalent to the number of writes to the register.

On the other hand, in the Winograd algorithm, the calculation time is 940 cycles, and it is clarified that the speed is 1.23 (= 1152/940) times faster than the examples of FIGS. 3 (a) to 3 (c).

Next, a computer that performs convolution calculation using such a Winograd algorithm will be described.

FIG. 5 is a hardware configuration diagram of a computer for performing convolution calculation in deep learning and the like.

As shown in FIG. 5, the computer 10 has a main memory 11 and a processor 12 connected via a bus 13.

Of these, the main memory 11 is a device that temporarily stores data such as DRAM (Dynamic Random Access Memory), and executes various programs in cooperation with the processor 12.

On the other hand, the processor 12 is hardware including an arithmetic unit such as an ALU (arithmetic and logic unit). In this example, DLU (Deep Learning Unit: registered trademark) is used as the processor 12. The DLU is a processor having an architecture suitable for deep learning, and has eight DPUs (Deep learning Processing Units) -chain 14.

FIG. 6A is a hardware configuration diagram of one DPU-chain 14.

As shown in FIG. 6A, the DPU-chain 14 includes four DPUs 15. Parallel calculation is performed on each of these DPUs 15 as described later.

Further, FIG. 6B is a hardware configuration diagram of one DPU15.

As shown in FIG. 6B, the DPU 15 has 16 DPEs (Deep learning Processing Elements) 0 to 15.
FIG. 7 is a hardware configuration diagram of each DPE.

As shown in FIG. 6B, the total number of DPEs is 16, but only DPE0 to DPE7 will be described below.

As shown in FIG. 7, each of DPE0 to DPE7 has eight calculation cores C # 0 to C # 7 and a register file 20 in which these calculation cores C # 0 to C # 7 can read and write.

Of these, the calculation cores C # 0 to C # 7 are independent SIMD (Single Instruction Multiple Data) arithmetic units, and parallel calculation can be executed in each calculation core C # 0 to C # 7.

On the other hand, the register file 20 is connected to the main memory 11 via the bus 13 (see FIG. 5), and stores the data read from the main memory 11 or the calculation cores C # 0 to C # 7. Stores the calculated calculation result.

In this example, the register file 20 is divided into four registers G # 0 to G # 3, and each of them can be read and written in parallel. For example, when the register G # 0 reads data from the main memory 11, the calculation results calculated by the calculation cores C # 0 to C # 7 can be stored in the register G # 1 in parallel with the data.

FIG. 8 is a hardware configuration diagram of DPE0.
Since DPE1 to DPE15 also have the same hardware configuration, the description thereof will be omitted. Further, FIG. 8 shows only the hardware configuration of the register G # 0 among the registers G # 0 to G # 3 of the register file 20. The other registers G # 1 to G # 3 have a similar hardware configuration.

As shown in FIG. 8, the register G # 0 includes eight banks R # 0 to R # 7. Each of the banks R # 0 to R # 7 is an example of a storage area, and is provided corresponding to each of the calculation cores C # 0 to C # 7. For example, bank R # 0 is a storage area provided corresponding to calculation core C # 0. When the calculation core C # 0 performs the calculation, the calculation core C # 0 reads the data in the bank R # 0, and the calculation core C # 0 writes the calculation result to the bank R # 0.

FIG. 9 is a diagram for explaining the line numbers assigned to each of the banks R # 0 to R # 7.

The line number is an identifier for identifying each entry of banks R # 0 to R # 7, and in this example, 128 line numbers of L _{0 to} L ₁₂₇ are used. The data stored in each entry is not particularly limited. In this example, floating point type data is stored in one entry. According to this, 127 floating point type data can be stored in bank R # 0. The same applies to banks R # 1 to R # 7.

Further, when the convolution calculation of deep learning is performed, the elements of the matrix to be the convolution calculation are stored in each entry. In that case, the matrix elements are stored as array elements in the main memory 11.

Therefore, next, an expansion method for expanding the array elements stored in the main memory 11 to DPE0 to DPE7 will be described.

There are sequential method and multicast method as the expansion method.
First, the sequential method will be described.

10 and 11 are schematic views for explaining the sequential method.

In this example, it is assumed that the array elements a [0], a [1], a [2], ... a [127] stored in the main memory 11 are expanded to DPE0 to DPE7.

In this case, first, as shown in FIG. 10A, the first array element a [0] is stored in the entry specified by the line number L ₀ in the bank R # 0 of DPE 0.

Next, as shown in FIG. 10B, the next array element a [1] is stored in the adjacent bank R # 1 without changing the line number L ₀ .

Then, as shown in FIG. 10 (c), the elements are stored one after another in the adjacent bank without changing the line number L ₀ . As a result, the entries specified by the line number L ₀ in each bank R # ₀ to R # 7 of DPE0 to DPE7 are array elements a [0], a [1], a [2], ... a [63]. Will be filled with.

Thereafter, as shown in FIG. 11 (a), stores the next array element a [64] in the entry specified by the line number L ₁ in the bank R # 0 of DPE0.

Then, as shown in FIG. 11 (b), without changing the line number L ₁ to the bank R # 1 next store the next array element a [65].

Furthermore, the array elements are stored one after another in the adjacent bank without changing the line number L ₁ in this way. As a result, as shown in FIG. 11 (c), the entry specified by the line number L ₁ in each bank R # 0 to R # 7 of DPE0 to DPE7 is the array element a [64], a [65], a. Filled with [66],… a [127].

From the above, the array elements a [0], a [1], a [2], ... a [127] are expanded to DPE0 to DPE7 by the sequential method. According to such a sequential method, entries that are in the same line number L _i of DPE0~DPE7 is going to be filled in the order, the next line number where the last entry in the line number L _i are filled L _{i + 1} Array elements are stored in.

Next, the multicast method will be described.
FIG. 12 is a schematic diagram for explaining the multicast method.

In this example, it is assumed that the array elements a [0], a [1], a [2], ... a [23] stored in the main memory 11 are expanded to DPE0 to DPE7.

In the multicast method, a [0], a [1], a [2], ... a [23] are stored in DPE0 in order. Then, in the same manner as this, a [0], a [1], a [2], ... A [23] are stored in each of DPE1 to DPE7. According to this method, the array elements stored in each of DPE0 to DPE7 are the same.

Next, the contents of the register when the convolution calculation is performed by the Winograd algorithm in the computer 10 will be described.

FIG. 13 is a diagram schematically showing the contents of the respective registers G # 0 of each DPE.

In the following, the symbol representing the matrix and the array containing each element of the matrix are represented by the same symbol. For example, an array that stores each element of the t × t bottom matrix d is represented by d, and an array that stores each element of the 3 × 3 weight matrix g is represented by g.

Then, these arrays d and g are described by the following equation (4).

In equation (4), N is the number of batches taking a value from 0 to 63. Cin is an input channel number that takes a value from 0 to 255, and Cout is an output channel number that takes a value from 0 to 383.

And H and W are variables that specify the elements in one bottom matrix. Similarly, H'and W'are variables that identify elements in a weight matrix.

In this case, the array d is expanded to the respective registers G # 0 of DPE0 to DPE7 by the sequential method.

In the case of a multiple array such as array d, it is stored in register G # 0 in order from the lowest array element. The lowest element of the array d is specified by the number of batches N. Therefore, array elements having batch numbers N of 0, 1, ... 7 are stored in the order of banks R # 0, R # 1, ... R # 7 of DPE0. Then, the array elements having batch numbers N of 8, 9, ... 15 are stored in order in banks R # 0, R # 1, ... R # 7 of DPE1. In this way, each element having a batch number N of 0 to 63 is expanded to DPE0 to DPE7.

In addition, in the array d [Cin] [H] [W] [N], the higher-order elements specified by Cin, H, and W are handled as follows.

First, as shown in FIG. 4A, the position where the small bottom matrix d of t × t is cut out from the bottom matrix is fixed, and the t × t elements of the small bottom matrix d are [H] [W]. Store in. Then, for Cin, the first 0 to 4 of the values from 0 to 255 is used.

According to this, the matrix elements of t × t corresponding to Cin = 0 are expanded in each of DPE0 to DPE7. Similarly, the t × t matrix elements corresponding to Cin = 1, Cin = 2, and Cin = 3 are also expanded to DPE0 to DPE7.

On the other hand, the array g is expanded to the respective registers G # 0 of DPE0 to DPE7 by the multicast method.

In this example, array elements with Cout values 0 to 7 are multicast for each input channel number Cin. For example, among the array elements whose Cout values are 0 to 7, the elements with Cin = 0 are multicast to each of DPE0 to DPE7. Similarly, the array elements of Cin = 0, Cin = 1, and Cin = 2 are also forwarded to DPE0 to DPE7 by multicast.

However, when the array g is transferred by the multicast method in this way, the values of the input channel number Cin and the output channel Cout in the bank R # 0 of DPE0 lose their regularity. This is inconvenient for the compute core C # 0 corresponding to the bank R # 0 to convolve the arrays g and d with the Winograd algorithm. The same applies to other calculation cores C # 1 to C # 7 and DPE1 to DPE7.
Therefore, the elements of the array g are rearranged as follows.

FIG. 14 is a diagram schematically showing the array elements of the array g in the main memory 11.

As described above, the array g is an array representing a weight matrix and can correspond to a 3 × 3 square matrix. Therefore, in the following, numbers 0, 2, ... 8 are assigned to each element of this 3 × 3 square matrix in order, and each element is identified by these numbers.

According to this, when g [Cout] [Cin] [H'] [W'] is described as in equation (4), 0, 2, ... 8 for each of [H'] and [W']. The number will be assigned.

FIG. 15 is a diagram showing the contents of register G # 0 of DPE0 immediately after being transferred by the above-mentioned multicast method.

As shown in FIG. 15, when forwarding is performed by the multicast method, one line of banks R # 0 to R # 7 is filled in order from the lower elements of g [Cout] [Cin] [H'] [W']. Be done. Then, when the last bank R # 7 of that line is filled, the next higher line is filled in order.

While the number of elements in the weight matrix g is 9, the number of banks R # 0 to R # 7 is 8, and the numbers do not match. Therefore, when the transfer is performed to the register by the multicast method in this way, the nine elements of Cin = 0 and Cout = 0 are stored in the register across the two lines. The same applies to other combinations of Cin and Cout.

As a result, array elements with various Cin and Cout values are stored in bank R # 0, and the regularity of Cin and Cout in bank R # 0 is reduced.

Therefore, in this example, each calculation core C # 0 to C # 7 of DPE0 buffers one of the remaining registers G # 1 to G # 3 of DPE0, and sets the elements of the array g in register G # 0. Align.

FIG. 16 is a diagram showing the contents of register G # 0 of DPE0 after alignment.

As shown in FIG. 16, by aligning, elements having the same Cout value are stored in the same bank. For example, bank R # 0 stores only elements with Cout = 0.

FIG. 17 is a diagram showing the contents of each register G # 0 of DPE0 to DPE7 after the alignment in this way.

As shown in FIG. 17, for example, the elements of the array g stored in the bank R # 0 of DPE0 are elements of Cout = 0 and Cin = 0 to 3. The elements of the array d stored in this bank R # 0 are elements of N = 0 and Cin = 0 to 3.

As a result, the Cin values of the arrays d and g in the bank R # 0 are the same, and the convolution of the arrays d and g having the same Cin values can be executed by the calculation core C # 0 according to the Winograd algorithm. ..

In addition, each bank R # 0 to R # 7 and the number of batches N have a one-to-one correspondence, and convolution calculation for different numbers of batches is executed in each bank R # 0 to R # 7. This also applies to other DPE1 to DPE7.

Then, it is expected that the forward processing and backward processing of deep learning can be executed at high speed by executing such convolution calculation in parallel by the calculation cores C # 0 to C # 7 of each DPE0 to DPE7.

However, as a result of the examination by the inventor of the present application, it has become clear that the method of making each bank R # 0 to R # 7 and the number of batches N have a one-to-one correspondence has the following problems.

FIG. 18 is a diagram for explaining the problem, and is a schematic diagram of bank R # 0 of register G # 0 of DPE0.

In this example, each bank R # 0 to R # 7 and the number of batches N have a one-to-one correspondence, and a small bottom matrix d and a weight matrix g having the same input channel number Cin are stored in one bank. .. Therefore, it is necessary to store the same number of small bottom matrix d and weight matrix in one bank, and if the size of the small bottom matrix d is increased, the elements of the small bottom matrix d overflow from the bank.

For example, consider a case where four small bottom matrices d and four weight matrices g are stored in bank R # 0 as shown in FIG. The size of the small bottom matrix d is t × t, and the size of the weight matrix g is 3 × 3. Therefore, the number of elements stored in bank R # 0 is 4 × t ² + 4 × 3 ² . As described above, the number of data that can be stored in one bank is 127, so t must be 4 or less in order to prevent the number of elements from exceeding this.

When t is small in this way, the size of the small top matrix y obtained by Eq. (2) is also small, so a large number of small top matrices y must be calculated in order to obtain the top matrix, and the calculation required for convolution. The time will be long. As a result, the feature of the Winograd algorithm, which can speed up the convolution calculation, cannot be fully utilized.

Hereinafter, each embodiment capable of executing the convolution calculation at high speed will be described.

(This Embodiment)
FIG. 19 is a hardware configuration diagram of the information processing device 31 according to the present embodiment.

The information processing device 31 is a computer such as a PC (Personal Computer) for generating a program that can be executed by the computer 10 (see FIG. 5), and includes a storage device 32, a main memory 33, a processor 34, an input device 35, and an input device 35. A display device 36 is provided. Each of these parts is connected to each other by a bus 37.

Of these, the storage device 32 is a secondary storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive), and stores the information processing program 39 according to the present embodiment.

By executing the information processing program 39, a program that can be executed by the computer 10 (see FIG. 5) can be generated as described later.

The information processing program 39 may be recorded on a computer-readable recording medium 38, and the processor 34 may be made to read the information processing program 39 of the recording medium 38.

Examples of such a recording medium 38 include a physically portable recording medium such as a CD-ROM (Compact Disc-Read Only Memory), a DVD (Digital Versatile Disc), and a USB (Universal Serial Bus) memory. Further, a semiconductor memory such as a flash memory or a hard disk drive may be used as the recording medium 38. These recording media 38 are not temporary media such as a carrier wave having no physical form.

Further, the information processing program 39 may be stored in a device connected to a public line, the Internet, a LAN (Local Area Network), or the like, and the processor 34 may read and execute the information processing program 39.

On the other hand, the main memory 33 is hardware that temporarily stores data such as DRAM (Dynamic Random Access Memory), and the information processing program 39 is deployed on the hardware.

The processor 34 is hardware such as a CPU (Central Processing Unit) that controls each part of its own device and executes an information processing program 39 in cooperation with a main memory 33.

The input device 35 is an input device such as a keyboard or a mouse operated by the user. Further, the display device 36 is a display device such as a liquid crystal display that displays various commands used by the user when the information processing program 39 is executed.

FIG. 20 is a functional configuration diagram of the information processing device 31 according to the present embodiment.
As shown in FIG. 20, the information processing device 31 has an output unit 41 and a calculation unit 42. Each of these parts is realized by the processor 34 and the main memory 33 collaborating to execute the above-mentioned information processing program 39.

Of these, the output unit 41 is a functional block that generates a program 50 that can be executed by the computer 10 (see FIG. 5). The program may be a file containing intermediate code or an executable binary file.

Further, the calculation unit 42 is a functional block that optimizes various parameters in the program 50. As the parameter, there is a size t of a small bottom matrix d cut out from the bottom matrix as shown in FIGS. 4A to 4C. In addition, the number q of banks, which will be described later, is also an example of parameters to be optimized.

FIG. 21 is a functional block diagram of the computer 10 realized by executing the program 50.

As shown in FIG. 21, the computer 10 includes a reception unit 51, a selection unit 52, a storage unit 53, a calculation unit 54, and an output unit 55. Each of these parts is realized by executing the program 50 in cooperation with the main memory 11 and the DLU 12 of FIG.

Of these, the reception unit 51 accepts inputs of the bottom matrix and the weight matrix. Further, as shown in FIGS. 4A to 4C, the selection unit 52 selects a small bottom matrix d of t × t from the bottom matrix. As described above, the value of the size t is optimized by the calculation unit 42, and the selection unit 52 selects the small bottom matrix d using the optimized size t.

Then, the storage unit 53 stores the elements of the small bottom matrix d and the weight matrix g in the banks R # 0 to R # 7 of DPE0 to DPE7.

Further, the calculation unit 54 performs a convolution calculation using these elements stored in the banks R # 0 to R # 7. The output unit 55 outputs a small top matrix y (see FIGS. 4A to 4C) which is the result of the convolution calculation.

Next, the function of the storage unit 53 will be described in detail.
The storage unit 53 is a functional block that stores the elements of each array read from the main memory 11 in each bank R # 0 to R # 7, but how to store them differs between forward processing and backward processing. ..

Here, the forward processing will be described. In the case of forward processing, the storage unit 53 rearranges the elements of each array read from the main memory 11 as shown in the following equation (5), and arranges each element into each bank R # 0 to R # 7 of DPE0 to DPE7. Store.

The array y is an array for storing the elements of the small top matrix obtained by convolving the small bottom matrix d and the weight matrix g. Then, in this example, the weight matrix g is an example of the first matrix, and the small bottom matrix d of t × t is an example of the second matrix.

Further, a (Cin number of) = (Cin number of _major) × (number of Cin _minor), it is possible to identify the input channel number Cin set (Cin _major, Cin _minor) by. Therefore, in the following, the set (Cin _major , Cin _minor ) and the input channel number Cin are equated. For example, an array element with Cin _major = 0 and Cin _minor = 0 corresponds to Cin = 0, and an array element with Cin _major = 0 and Cin _minor = 1 corresponds to Cin = 1.

Similarly, (number of N) = (number of N _major ) × (number of N _minor ), and the number of batches N can be specified by the set (N _major , N _minor ). _{Equalize major} , N _minor ) with the number of batches N. For example, the array elements of N _major = 0 and N _minor = 0 correspond to N = 0, and the array elements of N _major = 0 and N _minor = 1 correspond to N = 1.

According to the equation (5), one small bottom matrix d can be specified by specifying the input channel number Cin and the number of batches N. The input channel number Cin in this example is an example of the first identifier that identifies the small bottom matrix d in this way. Similarly, the number of batches N in this example is an example of a second identifier that identifies the small bottom matrix d.

In this example, the 4 total number of Cin _minor, and the total number of N _minor 16. Furthermore, the total number of Cin _major was 1, the total number of N _major is four. As a result, as shown in Fig. 2, 4 (= 1 × 4) of the 256 input channel number Cins in total and 64 (= 4 × 16) batch numbers are specified in the bottom matrix. On the other hand, the convolution calculation is performed.

Furthermore, the elements [H] [W] in the array d correspond to each element of the small bottom matrix d of t × t.

On the other hand, the elements [H'] [W'] of the array g correspond to each element of the weight matrix g of 3 × 3. The total number of input channel numbers Cin in the array g is four, which is the same as the input channel numbers in the array d. The total number of output channel number Couts is eight.

FIG. 22 is a diagram showing the contents of each register G # 0 of DPE0 to DPE7 in which the arrays d and g are stored by the storage unit 53 when the forward processing is performed.

In DPE0, each of the plurality of calculation cores performs a convolution calculation between the matrices d and g stored in the banks corresponding to the banks R # 0 to R # 7. Since the convolution calculation is executed in parallel in each of the plurality of calculation cores, the convolution calculation can be speeded up. This also applies to DPE0 to DPE7.

Further, among the arrays d and g, the array d is stored in each bank R # 0 to R # 7 of DPE0 to DPE7 in a sequential manner as in FIG. Here, only the array d having the same Cin _major is stored in each bank R # 0 to R # 7 at a time. Then, after the convolution of the array d is completed, Cin _major stores the array d having another value in each bank R # 0 to R # 7. FIG. 22 assumes a case where the array d of Cin _major = 0 is stored in each bank R # 0 to R # 7.

At this time, in the present embodiment, since Cin _minor is described at the bottom of the array d and N _minor is described at the top of the array d as in the equation (5), each bank and Cin _minor have the same range of N _minor. There is a one-to-one correspondence. Therefore, if the total number of Cin _minor q (= 4) pieces that, each of q banks in one DPE, input channel number (Cin _major, Cin _minor) is different from each other, and the number of batch (N _major , N _minor ) will store q small bottom matrices d with the same.

For example, in DPE0, the number of batches N is (0,0) and the input channel number Cin is (0,0) in each of the 4 (= q) banks of R # 0 to R # 3. Four small bottom matrices d, which are (0, 1), (0, 2), and (0, 3), are stored.

As a result, unlike the example in which the batch number N is changed for each bank R # 0 to R # 7 as shown in FIG. 13, the convolution calculation of q small bottom matrices d having the same batch number N is calculated as q. Cores can run in parallel.

On the other hand, the weight matrix g is stored in the storage unit 53 from the main memory 11 in each bank of DPE0 to DPE7 by a multicast method as in the example of FIG.

Here, the storage unit 53 stores the weight matrix g having the same input channel number Cin as the small bottom matrix d in each bank of DPE0 to DPE7. By storing the matrices d and g having the same input channel numbers Cin in the same bank in this way, the calculation unit 54 tells the matrices d and g having the same input channel numbers Cin to each other as shown in FIG. Convolution calculation can be performed.

However, when the array g is transferred to each bank by the multicast method, the regularity of the input channel number Cin and the output channel Cout in one bank is reduced as described with reference to FIG. Therefore, in the present embodiment, when the convolution calculation is performed by the Winograd algorithm, the calculation unit 54 aligns the elements of the array g as follows.

23 to 25 are diagrams showing the contents of each register G # 0 to G # 3 of DPE0 when the calculation unit 54 performs the convolution calculation by the Winograd algorithm. Note that, in FIGS. 23 to 25, only the bank R # 0 of the registers G # 0 to G # 3 is shown in order to avoid complicating the figure.

Before the convolution calculation is performed, as shown in FIG. 23A, the elements of the arrays d and g are stored in the bank R # 0 of the register G # 0. Of these, as the array d, as described above, a plurality of arrays d having different batch numbers N (= (N _major , N _minor )) are stored in bank R # 0.

Next, according to equation (2), the matrices B ^T and B are multiplied from both sides of the array d, and the resulting matrix B ^T dB is stored in the same line as the array d. The elements of the matrices B ^T and B are stored in the constant area cst of bank R # 0.

Further, at this stage, the array g representing the weight matrix is in a state in which the regularity is disturbed as shown in FIG.

Therefore, in the next step, as shown in FIG. 23B, each element of the array g stored in the bank R # 0 of the register G # 0 is transferred to the bank R # 0 of the register G # 3. While aligning each element.

As shown in FIG. 16, the contents of the aligned register have a one-to-one correspondence between banks R # 0 to R # 7 and the output channel number Cout, and bank R # 0 has an element of Cout = 0. Only stored.

Next, as shown in FIG. 24, according to equation (2), multiplied by the matrix G, G ^T from both sides of the sequence g, stores the the result matrix GGG ^T in the free space of the same bank. Each element of the matrix G and G ^T is stored in the constant area cst of bank R # 0.

Then, as shown in FIG. 25, the two matrices B ^T dB of the bank R # 0 of the register G # 0, with respect to a single matrix GDG ^T of the bank R # 0 of the register G # 3, the formula Perform the multiplication "◎" for each element in (2).

The convolution calculation is performed on two matrices having the same input channel number Cin as described with reference to FIG. Therefore, the matrix of Cin = 0 of the four matrices GdG ^{T in} bank R # 0 of register G # 3 and the two matrices B ^T dB of Cin _minor = 0 in bank R # 0 of register G # 0. Perform the multiplication "◎" for each element using.

After this, multiply the matrices A ^T and A from both sides of [GgG ^T ] ◎ [B ^T dB] according to Eq. (2) to obtain a small top matrix y.

As described above, the convolution calculation using the Winograd algorithm performed by the calculation unit 54 is completed.

According to such a convolution calculation, as shown in FIG. 23A, a bottom matrix having a different number of batches N (= (N _minor , N _major )) is stored in the bank R # 0 of the register G # 0. To do.

As a result, as compared with the example of storing a plurality of small bottom matrices d having the same number of batches N but different input channel numbers Cin in the same bank as shown in FIG. 17, the small bottom matrix d stored in one bank The number can be reduced. As a result, the size t of the bottom matrix d can be increased, and the convolution calculation can be performed at high speed by the Winograd algorithm.

As a result of a trial calculation by the inventor of the present application for the case of t = 6, in the examples of FIGS. 3A to 3C which do not use the Winograd algorithm, the calculation time required for convolution was 2304 cycles. On the other hand, in the present embodiment, the calculation time is 1264 cycles, and it has been clarified that the speed can be increased by 1.82 (= 2304/1264) times.

To perform the convolution calculation even faster, the value of t should be made as large as possible, but if t is made too large, it will not be possible to store the small bottom matrix d in each of banks R # 0 to R # 7. .. On the other hand, if the value of t is small, the small bottom matrix d can be reliably stored in each of the banks R # 0 to R # 7, but the calculation time of the convolution calculation becomes long.

Therefore, in the present embodiment, the optimum value of t is obtained as follows.
First, each parameter is defined as follows.
p: Number of banks in one DPE
q: Number of banks in which small bottom matrix d with the same N _minor is stored in one DPE
R: Number of data that can be stored in one bank

In the case of the example of FIG. 22, the specific values of these parameters are as follows.

p: 8
q: 4 pieces
R 128 pieces

In addition, the following parameters are defined.
Cin': Number of input channel numbers Cin processed by DPE0 at one time
Cout': Number of output channel numbers Cout to be processed by DPE0 at one time
N': Number of batches to be processed by DPE0 at one time Number of batches N These parameters will be described with reference to the example of FIG.

Cin'is the number of input channel number Cins processed at one time by DPE0 as described above. The input channel number Cin is specified as a set (Cin _major , Cin _minor ), but in the example of FIG. 22, (Cin _major , Cin _minor ) = (0, 0), (0, 1), (0, 2), Since only the arrays g and d of (0, 3) are processed by DPE0, Cin'= 4.

On the other hand, Cout'is the number of output channel number Couts processed at one time by DPE0 as described above. In the example of FIG. 22, since eight weight matrices g having Cout values 0 to 7 are stored in DPE0, Cout ′ = 8.

Further, N'is the number of batches N processed by DPE0 at one time as described above. In the example of FIG. 22, four small bottom matrices d in which the set (N _major , N _minor ) is (0, 0), (0, 1), (1, 0), (1, 1) are processed by DPE0. Therefore, N'= 4.
Next, the calculation time of the convolution calculation will be examined.

First, the calculation time when the matrix B ^T dB is obtained from the small bottom matrix d of t × t as shown in FIG. 23 (a) will be examined. To find the matrix B ^T dB, for example, first calculate B ^T d and then multiply the calculation result by the matrix B from the right. To calculate B ^T d, the small bottom matrix d of t × t can be decomposed into t column vectors, and the product of the column vector and the matrix B ^T can be obtained.

Therefore, in this example, the calculation time required to obtain the product of one of the t column vectors constituting the small bottom matrix d of t × t and the matrix B ^T is written as b (t). Using the function b (t), the calculation time required to obtain B ^T dB with one DPE can be written as the following equation (6).

The reason why "t" is included in the equation (6) is expressed by the function b (t) because it is necessary to multiply the matrix B ^T by t column vectors of the small bottom matrix d when finding B ^T d. This is in consideration of the fact that a calculation time that is t times longer than the calculation time is required. Similarly, when finding the product of the matrix B ^T d and B, it is necessary to multiply the matrix B ^T d by the t column vectors of the matrix B. Therefore, since the total calculation time is t + t times the calculation time represented by the function b (t), the factor "t + t" is included in the equation (6).

Further, as shown in FIG. 22, since one DPE has a total of Cin'・ N'small bottom matrix d, the number of small bottom matrices d per bank is Cin'・ N'/. There are q pieces. Since each of the calculation cores C # 0 to C # 7 needs to obtain B ^T dB for each of the Cin'・ N'/ q small bottom matrices d in one bank corresponding to itself, the equation ( Factors Cin'and N'/ q were included in 6).

Next, consider the computation time for obtaining the matrix GGG ^T from the 3 × 3 weight matrix g as shown in Figure 24.

To determine the matrix GGG ^T, for example first calculating the Gg,, multiply the matrix G ^T from the right to the calculation result. Further, in order to calculate Gg, the weight matrix g may be decomposed into three column vectors, and the product of the column vector and the matrix G may be obtained.

Therefore, in this example, the calculation time required to obtain the product of one of the three column vectors constituting the 3 × 3 weight matrix g and the matrix G is written as w (t). Using the function w (t), the calculation time required to obtain GgG ^T with one DPE can be written as the following equation (7).

The reason why "3" is included in the equation (7) is that when the matrix Gg is calculated, it is necessary to multiply the matrix G by the three column vectors of the weight matrix g, so the calculation time represented by the function w (t) This is in consideration of the fact that the calculation time is three times longer.

When finding the product of the matrix G g and the matrix G ^T , it is necessary to multiply the matrix G g by the t column vectors of the matrix G ^T. Therefore, since the total calculation time is t + 3 times longer than the calculation time represented by the function w (t), the factor "t + 3" is included in the equation (7).

Further, as shown in FIG. 22, since one DPE has a total of Cin'・ Cout' weight matrices g, the number of weight matrices g per bank is Cin'・ Cout' / p. It becomes. Since each of the calculation cores C # 0 to C # 7 needs to find GgG ^T for each of the Cin'・ Cout' / p small bottom matrices d in one bank corresponding to itself, the equation (7) ) Includes the factors Cin'and Cout' / p.

Next, as shown in FIG. 25, the calculation time required to multiply the matrix B ^T dB and G g G ^T for each element will be examined.

As shown in FIG. 22, the number of small bottom matrices d stored in one DPE is N'・ Cin' ・ Cout' / p. The number of elements in the small bottom matrix d is t ² . Therefore, the number of multiplications when multiplying each element of the matrices B ^T dB and G g G ^T is expressed by the following equation (8).

In equations (6) to (8), N'from the number of N batches was selected, Cout' was selected from the output channel numbers of Cout, and Cin' was selected from the input channel numbers of Cin. The calculation time for the case. Therefore, in order to perform the convolution calculation of all the bottom matrix and the weight matrix of FIG. 2, it is necessary to further perform the calculation as many times as the following equation (9).

The factor HW / (t-2) ² in Eq. (9) represents the total number of cutting methods when cutting out a submatrix of t × t from the bottom matrix of H × W.

According to the above equations (6) to (9), the calculation time depends not only on t but also on q. Therefore, in this embodiment, the calculation time when the convolution calculation is performed with one DPE is represented by the first function f (t, q). The first function f (t, q) can be expressed as the following equation (10) by multiplying the sum of equations (6) to (7) by equation (9).

To reduce the computational time required for convolution, the first function f (t, q), provided that the number of elements in each of the weight matrix g and the small bottom matrix d does not exceed the number that can be stored in the register. Find the combination of t and q that minimizes the value.

Therefore, next, the number of elements of each of the small bottom matrix d and the weight matrix g will be examined.
First, the number of elements of the small bottom matrix d will be described.

The number of elements E _b of the small bottom matrix d in one bank of one DPE can be expressed by the following equation (11).

In equation (11), t ² is the number of elements in one small bottom matrix d. Cin'・ N'/ q is the number of small bottom matrices d stored in one bank.

On the other hand, the number of elements E _w of the weight matrix g in one bank of one DPE can be expressed by the following equation (12).

In equation (12), 3 ² is the number of elements in one weight matrix g. Cin'and Cout' / p are the number of weight matrices g stored in one bank.

From equations (11) and (12), the second function g (t, q) representing the total number of elements of each of the small bottom matrix d and the weight matrix g can be written as the following equation (13).

Assuming that the number of data that can be stored in one bank is R as described above, the constraint condition of the following equation (14) can be obtained.

As described above, among the combinations of t and q that satisfy the constraint condition of the equation (14), the combination of t and q that minimizes the value of the first function f (t, q) of the equation (10) is selected. By finding it, the convolution calculation can be speeded up.

Therefore, in the present embodiment, the calculation unit 42 minimizes the value of the first function f (t, q) of the equation (10) among the combinations of t and q that satisfy the constraint condition of the equation (14). Calculate the combination of t and q like this.

In this embodiment, R = 128, and the number of candidates for t and q satisfying the equation (14) is not so large. Therefore, the calculation unit 42 finds a combination of t and q that satisfies the equation (14) in the full search, and minimizes the value of the first function f (t, q) of the equation (10) among them. Can be identified.

By the way, in equation (10), b (t) and w (t) are treated as known functions. b (t) and w (t) can be obtained as follows.

First, how to obtain w (t) will be described. As described above, w (t) is the calculation time required to calculate the product of one of the three column vectors constituting the 3 × 3 weight matrix g and the matrix G when calculating Gg. Is. When t = 6, each element of the matrix G becomes as shown in the following equation (15).

この行列Gは次の式（１６）のように変形できる。

This matrix G can be transformed by the following equation (16).

The two matrices on the right side of equation (16) are set as in equations (17) and (18) below.

Therefore, in order to calculate Gg, it is sufficient to calculate G'g first and then multiply the result by G "from the left. Therefore, the calculation method of G'g will be described.

In the following, one column g'of a 3 × 3 weight matrix g is written as (g ₀ , g ₁ , g ₂ ) ^T. Then, G'g'can be written as the following equation (19).

Note that (x ₀ , x ₁ , x ₂ , x ₃ , x ₄ , x ₅ ) ^T is a variable that stores each element of G'g'.

Here, in order to calculate the equation (19), six array elements a [0], a [1], a [2], a [3], a [4], and a [5] are prepared. To do. Then, g ₀ , g ₁ , and g ₂ are stored in each of a [0], a [1], and a [2]. Then, two array elements b [0] and b [1] are prepared as buffers for calculation.

At this time, the calculation of the equation (19) can be realized by assigning values to each array element in the order shown in FIG.

FIG. 26 is a schematic diagram showing the calculation of the equation (19) in step order. In addition, "//" in FIG. 26 is a comment sentence which expresses the meaning of each step. This also applies to FIG. 27, which will be described later.

When the calculation is performed according to the procedure shown in FIG. 26, finally (a [0], a [1], a [2], a [3], a [4], a [5]) = (x ₀ , x ₁ , x ₅ , x ₂ , x ₄ , x ₃ ), and each of the array elements a [0], a [1], a [2], a [3], a [4], a [5] The calculation result of G'g'can be stored in.

Then, the calculation of G'g'can be performed in 8 steps. Therefore, w (6) = 8. If the value of t is different from 6, the value of w (t) can be obtained in the same way.

Next, how to obtain b (t) will be described. As described above, b (t) is the calculation time required to obtain the product B ^T d of one of the t column vectors constituting the small bottom matrix d of t × t and the matrix B ^T. .. When t = 6, each element of the matrix B ^T becomes as shown in the following equation (20).

Further, one column d'of a 6 × 6 small bottom matrix d is written as (d ₀ , d ₁ , d ₂ , d ₃ , d ₄ , d ₅ ) ^T below. At this time, B ^T d'can be written as the following equation (21).

Note that (x ₀ , x ₁ , x ₂ , x ₃ , x ₄ , x ₅ ) ^T is a variable that stores each element of B ^T d'.

Here, in order to calculate the equation (21), six array elements a [0], a [1], a [2], a [3], a [4], and a [5] are prepared. Then, d ₀ , d ₁ , d ₂ , d ₃ , d ₄ , d ₅ are stored in each of them in advance.

Then, four array elements b [0], b [1], b [2], and b [3] are prepared as buffers for calculation.

At this time, the calculation of the equation (21) can be realized by substituting the values for each array element in the order shown in FIG.

FIG. 27 is a schematic diagram showing the calculation of the equation (21) in step order.
When the calculation is performed according to the procedure shown in FIG. 27, finally (a [0], a [1], a [2], a [3], a [4], a [5]) = (x ₀ , x ₁ , x ₂ , x ₃ , x ₄ , x ₅ ), and each of the array elements a [0], a [1], a [2], a [3], a [4], a [5] The calculation result of B ^T d'can be stored in.

Then, the calculation of B ^T d'can be performed in 15 steps. Therefore, b (6) = 15. When the value of t is different from 6, the value of b (t) can be obtained in the same manner.

Based on the matters described above, the information processing apparatus 31 according to the present embodiment executes the following information processing method.

FIG. 28 is a flowchart of the information processing method according to the present embodiment.
First, in step S1, the calculation unit 42 (see FIG. 20) calculates the combination of t and q. For example, the calculation unit 42 calculates the combination of t and q that satisfies the constraint condition of the equation (14) that minimizes the value of the first function f (t, q) of the equation (10). .. As a result, among the combinations of t and q that can store the elements of the small bottom matrix d of the weight matrix g and t × t in q banks, the combination with the minimum calculation time can be obtained.

Next, the process proceeds to step S2, and the output unit 41 (see FIG. 20) outputs a program 50 that can be executed by the computer 10 (see FIG. 5).

The combination of t and q calculated in step S1 is used for the program 50. For example, when the program 50 is executed on the computer 10, the selection unit 52 (see FIG. 21) selects a small bottom matrix d of t × t from the bottom matrix.

Then, the storage unit 53 stores the small bottom matrix d of t × t and the weight matrix g in each of the q banks of the banks R # 0 to R # 7 of DPE0. After that, the calculation unit 54 performs a convolution calculation of the small bottom matrix d and the weight matrix g using the Winograd algorithm according to the procedure of FIGS. 23 to 25.

This completes the basic steps of the information processing method according to this embodiment.

According to the present embodiment described above, the first function f (t) representing the calculation time of the convolution calculation under the constraint condition of the equation (14) that the small bottom matrix d and the weight matrix g can be stored in one bank. The calculation unit 42 calculates the combination of t and q that minimizes, q).

Therefore, while storing the small bottom matrix d and the weight matrix g in the register bank, it is possible to perform convolution calculation at high speed using these matrices.

<Backword processing>
In the example of FIG. 22, the convolution calculation in the forward processing of deep learning was performed by the Winograd algorithm.

The Winograd algorithm in the backward processing of deep learning will be described below. The backword process includes a process of convolving the top matrix and the weight matrix to obtain the bottom matrix, and a process of convolving the top matrix and the bottom matrix to obtain the weight matrix.

First, the process of obtaining the bottom matrix by the convolution calculation of the top matrix and the weight matrix as in the former will be described.

FIGS. 29 (a) to 29 (c) are schematic diagrams when the convolution calculation of the top matrix and the weight matrix is performed by the Winograd algorithm in the backward processing.

First, as shown in FIG. 29 (a), the selection unit 52 (see FIG. 21) selects a small top matrix y of t × t from the top matrix of H rows and W columns.

Next, according to the following equation (22), the calculation unit 54 obtains the small bottom matrix d by convolving the weight matrix g and the small top matrix y.

Next, as shown in FIG. 29 (b), the position where the small top matrix y is cut out from the top matrix is shifted by two columns from the case of FIG. 29 (a), and the same calculation as above is performed for the cut out small top matrix y. I do. The small bottom matrix d thus obtained forms a block next to the small bottom matrix d obtained in FIG. 29 (a) in the bottom matrix.

By shifting the positions for cutting out the small top matrix y from the top matrix by two in the column direction and the row direction in this way, a bottom matrix formed by each small bottom matrix d is obtained as shown in FIG. 29 (c). be able to.

This completes the convolution calculation of the top matrix and the weight matrix in the backword processing. In this example, the weight matrix g is an example of the first matrix, and the small top matrix y of t × t is an example of the second matrix.

Next, the function of the storage unit 53 in the case of performing the backword processing in this way will be described in detail.

The storage unit 53 arranges the elements of each array as shown in the following equation (23), and stores each element in each bank R # 0 to R # 7 of DPE0 to DPE7.

Here, assuming that N is the number of batches, (the number of N) = (the number of N _major ) × (the number of N _minor ), (the number of Cout) = (the number of Cout _major ) × (the number of Cout _minor ). is there. In this case, the number of batches N is specified by the set (N _major , N _minor ) as in the equation (5). In this backward processing, the batch number N is an example of a second identifier that identifies the small top matrix y.

The output channel number Cout is also specified as a pair (Cout _major , Cout _minor ). For example, an array element with Cout _major = 0 and Cout _minor = 0 corresponds to Cout = 0, and an array element with Cout _major = 0 and Cout _minor = 1 corresponds to Cout = 1. Then, in this backward processing, the output channel number Cout becomes the first identifier for identifying the small top matrix y.

Further, in this example, the total number of batches N is 64 and the total number of output channel numbers Cout is 384 as in FIG. Then, _let the total number of N _{majors be} 16, the total number of N _{minors be} 4, and the total number of Cout _{minors be} 4.

Further, the elements [H ″] [W ″] in the array y correspond to each element of the small top matrix y of t × t.

FIG. 30 is a diagram showing the contents of each register G # 0 of DPE0 to DPE7 in which the arrays y and g are stored by the storage unit 53.

Of the arrays y and g, the storage unit 53 stores the array y in each bank R # 0 to R # 7 of DPE0 to DPE7 in a sequential manner.

At this time, in the present embodiment, since Cout _minor is described at the bottom of the array y and N _minor is described at the top of the array y as in equation (23), each bank and Cout _minor have the same range of N _minor. There is a one-to-one correspondence. Therefore, assuming that the total number of Cout _minors is q (= 4), the output channel numbers (Cout _major , Cout _minor ) are different from each other for each of the q banks in one DPE, and the number of batches (N _major). , N _minor ) will store q small top matrices y with the same.

For example, in DPE0, the number of batches N is (0, 0) and the output channel number Cout is (0, 0), (0, 1) in each of the four banks R # 0 to R # 3. ), (0, 2), (0, 3), four small top matrices y are stored.

As a result, unlike the example in which the batch number N is changed for each bank R # 0 to R # 7 as shown in FIG. 13, the convolution calculation of q small top matrices y having the same batch number N is calculated as q. Can be run in parallel on the core.

On the other hand, regarding the weight matrix g, the storage unit 53 transfers the weight matrix g from the main memory 11 to each DPE0 to DPE7 by a multicast method as in the example of FIG.

As described with reference to FIG. 15, in the multicast method, there is no regularity in the values of the input channel number Cin and the output channel Cout. Therefore, also in this example, the calculation unit 54 aligns the array g in the same manner as in FIGS. 23 to 25.

Next, the calculation time of the convolution calculation in this backward processing will be examined.

The calculation time required to obtain B ^T yB of equation (22) with one DPE can be written as the following equation (24) by replacing Cin'in equation (6) with Cout'.

The calculation time required to determine the GGG ^T in one of DPE of formula (22), for the same reason as the formula (7) can be written as equation (25).

Further, in the equation (22), the number of multiplications when multiplying each element of the matrices B ^T y B and G g G ^T is expressed by the following equation (26) as in the equation (8).

Then, in order to perform the convolution calculation of all the top matrix and the weight matrix, it is necessary to perform the calculation as many times as the next equation (27) in which p in the equation (9) is replaced with Cout'.

The first function f (t, q), which expresses the calculation time when performing convolution calculation with one DPE, is the following formula (28) by multiplying the sum of formulas (24) to (26) by formula (27). ) Can be expressed as.

Next, consider the condition that the number of elements of each of the small top matrix y and the weight matrix g does not exceed the number that can be stored in the register.
First, the number of elements of the small top matrix y will be described.

The number of elements E _y of the small top matrix y in one bank of one DPE can be expressed by the following equation (29) by replacing Cin'in equation (11) with Cout'.

On the other hand, the number of elements E _w of the weight matrix g in one bank of one DPE can be expressed by the following equation (30) as in equation (12).

From equations (29) and (30), the second function g (t, q) representing the total number of elements including the small top matrix y and the weight matrix g can be written as the following equation (31).

Therefore, assuming that the number of data that can be stored in one bank is R, the constraint condition of the following equation (32) can be obtained.

As described above, among the combinations of t and q that satisfy the constraint condition of the equation (32), the combination of t and q that minimizes the value of the first function f (t, q) of the equation (28) is selected. By finding it, the convolution calculation can be speeded up.

Therefore, when performing backward processing to obtain a small bottom matrix d by convolving the top matrix and the weight matrix as in this example, the calculation unit 42 satisfies the constraint conditions of the equation (32) of t and q. Identify the combination. Then, among the specified combinations, the calculation unit 42 calculates the combination of t and q that minimizes the value of the first function f (t, q) of the equation (28), and speeds up the convolution calculation. ..

Next, the backward processing for convolving the top matrix and the bottom matrix to obtain the weight matrix will be described.

31 to 32 are schematic diagrams when the convolution calculation of the top matrix and the bottom matrix is performed by the Winograd algorithm in the backword processing.

First, as shown in FIG. 31A, the selection unit 52 selects a small top matrix y of t ′ × t ′ from the top matrix of H × W.

Then, as shown in FIG. 31 (b), the selection unit 52 selects the small bottom matrix d of (t'-2) x (t'-2) from the bottom matrix of H'x W'.

Subsequently, as shown in FIG. 32 (a), the calculation unit 54 selects the matrix y'of (t'-2) x (t'-2) from the small top matrix y. Then, the calculation unit 54 obtains 11 components of the weight matrix g according to the following equation (33).

Next, as shown in FIG. 32 (b), the position for selecting the matrix y'from the small top matrix y is shifted by one column from the case of FIG. 32 (a), and the calculation unit 54 with respect to the selected matrix y'. Finds the 12 components of the weight matrix g by performing the same calculation as above.

By shifting the position of cutting out the matrix y'from the small top matrix y in the column direction and the row direction in this way, each element of the 3 × 3 weight matrix g can be obtained as shown in FIG. 32 (c). ..

This completes the convolution calculation of the top matrix and bottom matrix in the backword process. In this example, the small bottom matrix d of (t'-2) × (t'-2) is an example of the first matrix, and the small top matrix y of t'× t'is an example of the second matrix. ..

Next, the function of the storage unit 53 when performing this backword processing will be described in detail.

The storage unit 53 arranges the elements of each array as shown in the following equation (34), and stores each element in each bank R # 0 to R # 7 of DPE0 to DPE7.

In this example as well, the small bottom matrix d is specified by the combination of the batch number N (= (N _major , N _minor )) and the input channel number Cin (= (Cin _major , Cin _minor )). The batch number N (= (N _major , N _minor )) is an example of the first identifier, and the input channel number Cin (= (Cin _major , Cin _minor )) is an example of the second identifier.

FIG. 33 is a diagram showing the contents of each register G # 0 of DPE0 to DPE7 in which the arrays y and d are stored by the storage unit 53.

The storage unit 53 stores the array d in each bank R # 0 to R # 7 of DPE0 to DPE7 in a sequential manner.

At this time, in the present embodiment, since N _minor is described at the bottom of the array d and Cin _minor is described at the top of the array d as in equation (34), each bank and N _minor have the same range of Cin _minor. There is a one-to-one correspondence. Therefore, _{assuming that} the total number of N _minors is q (= 4), the number of batches (N _major , N _minor ) is different from each other for each of the q banks in one DPE, and the input channel number (Cin _major). , Cin _minor ) will be stored as q small bottom matrices d.

For example, in DPE0, the input channel number Cin is (0, 0) and the number of batches N is (0, 0), (0, 1) in each of the four banks R # 0 to R # 3. ), (0, 2), (0, 3), four small bottom matrices d are stored.

As a result, unlike the example in which the number of batches N is changed for each bank R # 0 to R # 7 as shown in FIG. 13, the convolution calculation of q small bottom matrices d having the same input channel number Cin is performed by q. It can be executed in parallel on the compute core.

Further, for the small top matrix y, the storage unit 53 transfers the small top matrix y from the main memory 11 to each DPE0 to DPE7 by a multicast method.

Unlike the example of FIG. 30, in this example, Cout _minor is described at the bottom of the array y and N _minor is described at the top of the array y as shown in equation (34). The total number of Cout _minors is 4, and the total number of N _minors is 4.

As a result, for example, in DPE0, among the elements of the array y of N _major = 0 and N _minor = 0, the elements having the smallest value of Cout _minor are stored in banks R # 0 to R # 3 in order. Then, in banks R # 4 to R # 7, elements of N _major = 0 and N _minor = 1 are stored in ascending order of the value of Cout _minor .

Also, the elements of N _major = 1 in the array y are stored in banks R # 0 to R # 3 in order from the element with the smallest value of Cout _minor , and the values of N _minor are stored in banks R # 0 to R # 3. The element that is moved up by one is stored.

As a result, the elements of the array y having the same Cout _minor value are stored in one bank, so that it is not necessary to align each element of the array y in order to align the Cout _minor values in the bank.

Next, the calculation time of the convolution calculation in this backward processing will be examined.

The calculation time required to obtain Gy'G ^T in Eq. (33) with one DPE can be written as the following Eq. (35) by replacing t in Eq. (24) with t'.

In addition, the calculation time required to obtain the B ^T dB of equation (33) with one DPE is as follows: replace 3 in equation (25) with t'-2, replace t with t', and replace cout'with N'. By substituting, it can be written as the following equation (36).

Further, in the equation (33), the number of multiplications when multiplying each element of the matrix Gy'G ^T and the matrix B ^T dB is expressed by the following equation (37) as in the equation (8). ..

Then, in order to perform the convolution calculation of all the top matrix and the weight matrix, it is necessary to perform the calculation as many times as the following equation (38) as in the equation (27).

The first function f (t, q), which expresses the calculation time when performing convolution calculation with one DPE, is the following formula (39) by multiplying the sum of formulas (35) to (37) by formula (38). ) Can be expressed as.

Next, consider the condition that the number of elements of each of the small bottom matrix d and the small top matrix y does not exceed the number that can be stored in the register.

First, the number of elements of the small top matrix y will be described. The number of elements E _y of the small top matrix y in one bank of one DPE can be written as the following equation (40).

In equation (40), t ² is the number of elements in one small top matrix y. N'・ Cin'/ p is the number of small top matrices y stored in one bank.

On the other hand, the number of elements E _d of the small bottom matrix d in one bank of one DPE can be written as the following equation (41).

In equation (41), (t'-2) ² is the number of elements in one small bottom matrix d. N'・ Cout' / p is the number of small bottom matrices d stored in one bank.

From equations (29) and (30), the second function g (t, q) representing the total number of elements including the small top matrix y and the weight matrix g can be written as the following equation (42).

Therefore, assuming that the number of data that can be stored in one bank is R, the constraint condition of the following equation (43) can be obtained.

As described above, among the combinations of t and q that satisfy the constraint condition of the equation (43), the combination of t and q that minimizes the value of the first function f (t, q) of the equation (39) is selected. By finding it, the convolution calculation can be speeded up.

Therefore, when performing backward processing to obtain a weight matrix by convolving the bottom matrix and the top matrix as in this example, the calculation unit 42 sets a combination of t and q that satisfies the constraint condition of the equation (43). Identify. Then, among the specified combinations, the calculation unit 42 calculates the combination of t and q that minimizes the value of the first function f (t, q) of the equation (39), and speeds up the convolution calculation. ..

<1x1 convolution>
In deep learning, 1x1 convolution may be performed. For example, in ResNet-50 and ResNet101, 1x1 convolution is used. Therefore, the 1 × 1 convolution in the present embodiment will be described.

The matrix to be convolved by 1 × 1 is not particularly limited, but the convolution of the small bottom matrix d and the weight matrix g will be described below.

When convolving 1 × 1 of the matrices d and g, the storage unit 53 stores the elements of each matrix in an array as shown in the following equation (44), and stores each element in each bank R # of DPE0 to DPE7. Store in 0 to R # 7.

The order of the elements of the arrays d and g in the equation (44) is the same as that in the equation (5). For example, in the array d, Cin _minor is described at the bottom, and N _minor is described above it.

FIG. 34 is a diagram showing the contents of the register G # 0 of DPE0 in which the arrays d and g are stored by the storage unit 53 when 1 × 1 convolution is performed.

In the case of equation (5), the array d is stored in DPE0 to DPE7 by the sequential method as shown in FIG. 22, but in this example, the array d is stored in DPE0 to DPE7 by the multicast method.

As a result, for example, the elements of N _major = 0 and N _minor = 0 are stored in banks R # 0, R # 1, R # 2, R # 3 in the order of Cin _minor = 0,1,2,3. .. Then, when all the elements of N _major = 0 and N _minor = 0 are stored, the next element of N _major = 0 and N _minor = 1 is bank R # in the order of Cin _minor = 0,1,2,3. It is stored in 4, R # 5, R # 6, and R # 7. As a result, the first line of each bank R # 0 to R # 7 is filled, so the elements after N _minor = 2 are stored in the line one level above.

Note that the elements of the array d with N _major = 1 are expanded to DPE0 after the convolution of the elements with N _major = 0 is completed. The same applies to the elements of the array d in which the value of N _major is 2 or more.

Also, for the array g, the array d is stored in the bank R # 0 by the multicast method.

There is no Winograd algorithm applicable to 1x1 convolution. Therefore, in this example, the calculation unit 54 performs convolution according to the procedure shown in FIGS. 3 (a) to 3 (c) using the elements stored in each of the banks R # 0 to R # 7.

<Batch normalization>
In deep learning, performance may be improved by performing batch normalization. Batch normalization is a standardization method in which the average value of the pixel data of each image is set to 0 and the variance is set to 1 when the values of the pixel data are significantly different between the plurality of images. The method will be described below.

When batch normalization is performed, the storage unit 53 arranges each element of each array d and y as shown in the following equation (45), and multicasts each element to each bank R # 0 to R # 7 of DPE0 to DPE7. Store by method.

batch normalization can be applied to both bottom and top matrices. The case where batch normalization is performed on the small bottom matrix d, which is a part of the bottom matrix, will be described below.

FIG. 35 is a diagram showing the contents of the register G # 0 of DPE0 in which the small bottom matrix d is stored by the storage unit 53 at the time of batch normalization.

In this example, as in FIG. 34, the storage unit 53 stores the small bottom matrix d in the bank R # 0 by the multicast method. As shown in equation (45), Cin _minor is described at the bottom of the small bottom matrix d. Therefore, focusing on one of the banks R # 0 to R # 7, elements with the same Cin _minor value are stored in that bank. For example, bank R # 0 stores only elements with Cin _minor = 0.

Further, according to the equation (45), N _minor is described above Cin _{minor in} the small bottom matrix d. Therefore, focusing on one of the banks R # 0 to R # 7, elements with different batch numbers (N _major , N _minor ) are stored in that bank. For example, in bank R # 0, (N _major , N _minor ) = (0, 0), (0, 2), ... (0, 14), (1, 0), (1, 2), ... ( Elements of 1, 14), ... (3, 0), (3, 2), ... (3, 14) are stored.

In this way, one bank stores elements with the same Cin _minor but different batch numbers (N _major , N _minor ). Therefore, each of the calculation cores C # 0 to C # 7 uses only one bank corresponding to itself, and the average of multiple elements with the same Cin _minor but different batch numbers (N _major , N _minor ), and The variance of these elements can be calculated.

The calculation is executed by the calculation unit 54 as follows.
36 (a) and 36 (b) are diagrams showing the contents of register G # 0 of DPE0 for explaining the calculation performed by the calculation unit 54 at the time of batch normalization.

First, as shown in FIG. 36 (a), the calculation core C # 0 adds the values of each element of the small bottom matrix d in the bank R # 0, and the value x ₀ obtained by this is added to the bank R #. Store in line L _{sum_1} of 0. In the other banks R # 1 to R # 7, each of the calculation cores C # 1 to C # 7 adds the values of each element of the small bottom matrix d in the corresponding bank, and the value obtained by this is added. Store x _{1 to} x 7 in line L _{sum_1} of banks R # _{1 to} R # ₇ , respectively.

Here, as shown in FIG. 35, only the elements having an even N _minor are stored in the bank R # 0. Therefore, the value x ₀ is not the sum of the elements over all batch numbers (N _major , N _minor ), but the sum of only the values of the elements with even N _minor .

Therefore, the calculation unit 54 adds the values x _{0 to} x ₇ corresponding to the same Cin _minor . For example, since the value x ₀ and the value x ₄ both correspond to Cin _minor = 0, the calculation unit 54 adds both and writes the result in the value x ₀ . The value x ₀ obtained as a result is the sum of the elements of Cin _minor = 0 over all batch numbers (N _major , N _minor ). Similarly, the calculation unit 54 performs the following calculation.
x ₁ = x ₁ + x ₅
x ₂ = x ₂ + x ₆
x ₃ = x ₃ + x ₇

Next, the calculation core C # 0 calculates the mean value m ₀ by dividing the value x ₀ stored in the bank R # 0 by the number of batches, and sets the mean value m ₀ to the line L _mean of the bank R # 0. Store. In banks R # 1 to R # 3, each of the calculation cores C # 1 to C # 3 calculates the mean value m _{1 to} m ₃ of the values x _{1 to} x ₃ , and these values are used as banks R # 1 respectively. ~ Store in line L _mean of R # 3.

From the above, the average values m _{0 to} m ₃ of the elements of the small bottom matrix d are obtained for each bank R # 0 to R # 3.
Next, a calculation method for obtaining the variance will be described.

First, as shown in FIG. 36 (b), the calculation core C # 0 squares the values of each element of the small bottom matrix d in the bank R # 0, and the sum of the values obtained thereby y. ₀ is stored in the line _L sum_2 of the bank R # 0. In the other banks R # 1 to R # 7, each of the calculation cores C # 1 to C # 7 squares each element in the corresponding bank and adds them, resulting in the value y ₁ Store ~ y 7 in line L _{sum_2} of banks R # 1 to R # ₇ , respectively.

Similar to the example in FIG. 36 (a), the value y ₀ is not the sum of the squares of the elements over all batch numbers (N _major , N _minor ), but only the squared values of the elements where N _minor is an even number. It becomes a thing. Therefore, the calculation unit 54 writes the sum of the squares of the elements of the small bottom matrix d over all the batch numbers (N _major , N _minor ) in each of the values y ₀ to y ₃ by performing the following calculation.
y ₀ = y ₀ + y ₄
y ₁ = y ₁ + y ₅
y ₂ = y ₂ + y ₆
y ₃ = y ₃ + y ₇

Next, the calculation core C # 0 calculates the mean value a ₀ by dividing the value y ₀ stored in the bank R # 0 by the number of batches, and _{puts the} mean value a ₀ into the line L _{mean_2} of the bank R # 0. Store. In banks R # 1 to R # 3, each of the calculation cores C # 1 to C # 3 calculates the mean value a _{1 to} a ₃ of the values y _{1 to} y ₃ , and these values are used as banks R # 1 respectively. ~ Store in line L _{mean_2} of R # 3.

Thus, so that the bank R # 0 to R # mean value of the square of the elements of the small bottom matrix d every ₃ a ₀ ~a 3 was obtained.

Next, the calculation unit 54 calculates the variance v ₀ of each element of the small bottom matrix d in the bank R # 0 by calculating v ₀ = a ₀ -m ₀ ^2, and uses it as the bank R # 0. Store in the line L _var of. Similarly, the calculation unit 54 calculates the variance v ₁ to v ₃ of each element of the bank R # 1 to R # 3 by performing the following calculation, it banks R # 1 to R # 3 Line Store in L _var .
v ₁ = a ₁ -m ₁ ²
v ₂ = a ₂ -m ₂ ²
v ₃ = a ₃ -m ₃ ²

After this, the calculation unit 54 uses the values of each element of the small bottom matrix d (d [N _major ] [Cin _major ] [H] [W] [N _minor ] [i] as shown in the following equation (46). ) and by dividing the difference between the average value m _i in a distributed v _i, performs batch normalization for elements of _{Cin minor = i (i = 0,1,2,3} ).

以上によりbatch normalizationを終える。

This completes batch normalization.

By performing batch normalization in this way, improvement in learning performance in deep learning can be expected.

The following additional notes will be further disclosed with respect to each of the above-described embodiments.
(Appendix 1) The total number of elements of each of the plurality of first matrices and the plurality of second matrices of t rows and t columns is the number of data that can be stored in each of q of the plurality of storage areas of the register. Of the combinations of t and q that do not exceed, each of the q calculation cores corresponding to each of the q storage areas is a convolution calculation of the plurality of the first matrix and the plurality of the second matrix. A calculation unit that calculates the combination that minimizes the calculation time when executing in parallel with the Winograd algorithm,
A process of storing a plurality of the first matrix and a plurality of the second matrix of t rows and t columns in each of the q storage areas using the calculated combination of t and q, and q of the above. Outputs a program for causing a computer having the calculation core and the register to perform a process in which each of the calculation cores performs a convolution calculation of the first matrix and the second matrix using the Winograd algorithm. Output section and
An information processing device characterized by having.
(Supplementary note 2) The information processing apparatus according to Supplementary note 1, wherein each of the first matrix and the second matrix is a matrix in a convolutional layer of deep learning.
(Appendix 3) The calculation time is represented by the first function f (t, q), and each of the plurality of the first matrix and the plurality of the second matrix stored in the storage area. When the total number of elements is represented by the second function g (t, q), the calculation unit calculates the number of data that can be stored in one storage area by the second function g (t, q). The information processing apparatus according to Appendix 1, wherein the combination of q and t that minimizes the value of the first function f (t, q) is calculated within a range in which the value of is not exceeded.
(Appendix 4) Each of the first matrix and the second matrix is a matrix in the convolutional layer of deep learning.
The first function f (t, q) and the second function g (t, q) in the backward processing of the deep learning are the first function f (t, q) in the forward processing of the deep learning. ) And the information processing apparatus according to Appendix 3, which is different from the second function g (t, q).
(Appendix 5) Each of the plurality of the second matrices is specified by a combination of the first identifier and the second identifier.
The program
To have the computer execute a process of storing each of the q second matrices having the same second identifier in each of the q storage areas, which are different from each other in the first identifier. The information processing apparatus according to Appendix 1.
(Appendix 6) The program is
The first matrix and the second matrix having the same first identifiers are stored in the same storage area.
The information processing apparatus according to Appendix 5, wherein the computer is made to execute a process of executing the convolution calculation between the first matrix and the second matrix stored in the same storage area. ..
(Appendix 7) The program is
The mean value and the variance of the value of the element are calculated for each of the plurality of storage areas.
Addendum 1 characterized in that the computer is made to execute a process of normalizing the value of the element by dividing the difference between the value of the element and the average value by the variance for each of the plurality of storage areas. The information processing device described.
(Appendix 8) The total number of elements of each of the plurality of first matrices and the plurality of second matrices of t rows and t columns is the number of data that can be stored in each of q of the plurality of storage areas of the register. Of the combinations of t and q that do not exceed, each of the q calculation cores corresponding to each of the q storage areas is a convolution calculation of the plurality of the first matrix and the plurality of the second matrix. To calculate the combination that minimizes the calculation time when executing in parallel with the Winograd algorithm, and
A process of storing a plurality of the first matrix and a plurality of the second matrix of t rows and t columns in each of the q storage areas using the calculated combination of t and q, and q of the above. Outputs a program for causing a computer having the calculation core and the register to perform a process in which each of the calculation cores performs a convolution calculation of the first matrix and the second matrix using the Winograd algorithm. Processing to do and
An information processing program that allows a computer to execute.
(Appendix 9) The total number of elements of each of the plurality of first matrices and the plurality of second matrices of t rows and t columns is the number of data that can be stored in each of q of the plurality of storage areas of the register. Of the combinations of t and q that do not exceed, each of the q calculation cores corresponding to each of the q storage areas is a convolution calculation of the plurality of the first matrix and the plurality of the second matrix. To calculate the combination that minimizes the calculation time when executing in parallel with the Winograd algorithm, and
A process of storing a plurality of the first matrix and a plurality of the second matrix of t rows and t columns in each of the q storage areas using the calculated combination of t and q, and q of the above. Outputs a program for causing a computer having the calculation core and the register to perform a process in which each of the calculation cores performs a convolution calculation of the first matrix and the second matrix using the Winograd algorithm. Processing to do and
An information processing method characterized by a computer executing.

10 ... Computer, 11 ... Main memory, 12 ... Processor, 13 ... Bus, 20 ... Register file, 21 ... Information processing program, 31 ... Information processing device, 32 ... Storage device, 33 ... Main memory, 34 ... Processor, 35 ... Input device, 36 ... Display device, 37 ... Bus, 38 ... Recording medium, 39 ... Information processing program, 41 ... Output unit, 42 ... Calculation unit, 50 ... Program, 51 ... Reception unit, 52 ... Selection unit, 53 ... Storage Unit, 54 ... Calculation unit, 55 ... Output unit.

Claims (5)

The total number of elements of each of the plurality of first matrices and the plurality of second matrices of t rows and t columns does not exceed the number of data that can be stored in each of q of the plurality of storage areas of the register. Of the combinations of and q, each of the q calculation cores corresponding to each of the q storage areas performs a convolution calculation of a plurality of the first matrix and a plurality of the second matrices by the Winograd algorithm. A calculation unit that calculates the combination that minimizes the calculation time when executing in parallel,
A process of storing a plurality of the first matrix and a plurality of the second matrix of t rows and t columns in each of the q storage areas using the calculated combination of t and q, and q of the above. Outputs a program for causing a computer having the calculation core and the register to perform a process in which each of the calculation cores performs a convolution calculation of the first matrix and the second matrix using the Winograd algorithm. Output section and
An information processing device characterized by having. The information processing apparatus according to claim 1, wherein each of the first matrix and the second matrix is a matrix in a convolutional layer of deep learning. Each of the plurality of second matrices is identified by a combination of a first identifier and a second identifier.
The program
To have the computer execute a process of storing each of the q second matrices having the same second identifier in each of the q storage areas, which have different first identifiers from each other. The information processing apparatus according to claim 1. The total number of elements of each of the plurality of first matrices and the plurality of second matrices of t rows and t columns does not exceed the number of data that can be stored in each of q of the plurality of storage areas of the register. Of the combinations of and q, each of the q calculation cores corresponding to each of the q storage areas performs a convolution calculation of a plurality of the first matrix and a plurality of the second matrices by the Winograd algorithm. The process of calculating the combination that minimizes the calculation time when executing in parallel, and
A process of storing a plurality of the first matrix and a plurality of the second matrix of t rows and t columns in each of the q storage areas using the calculated combination of t and q, and q of the above. Outputs a program for causing a computer having the calculation core and the register to perform a process in which each of the calculation cores performs a convolution calculation of the first matrix and the second matrix using the Winograd algorithm. Processing to do and
An information processing program that allows a computer to execute. The total number of elements of each of the plurality of first matrices and the plurality of second matrices of t rows and t columns does not exceed the number of data that can be stored in each of q of the plurality of storage areas of the register. Of the combinations of and q, each of the q calculation cores corresponding to each of the q storage areas performs a convolution calculation of a plurality of the first matrix and a plurality of the second matrices by the Winograd algorithm. The process of calculating the combination that minimizes the calculation time when executing in parallel, and
A process of storing a plurality of the first matrix and a plurality of the second matrix of t rows and t columns in each of the q storage areas using the calculated combination of t and q, and q of the above. Outputs a program for causing a computer having the calculation core and the register to perform a process in which each of the calculation cores performs a convolution calculation of the first matrix and the second matrix using the Winograd algorithm. Processing to do and
An information processing method characterized by a computer executing.

Images