KR20220137443A - Method, apparatus and storage for storing a program for multi-demensional matrix multiplication - Google Patents

Abstract

The embodiments relate to a method, operation device, and program for multiplying a multidimensional matrix for improving a multiplication operation processing performance for a large multidimensional matrix, wherein a result matrix can be derived through multiplication between the compressed bit vectors. The method comprises: a step of outputting as a plurality of binarized matrices; a step of converting as a plurality of compressed bit vectors; a step of calculating a position of a specific bit; a step of outputting a result matrix; and a step of transforming the result matrix.

Description

A storage medium for storing a compressed multidimensional matrix multiplication method, an arithmetic unit and a program {METHOD, APPARATUS AND STORAGE FOR STORING A PROGRAM FOR MULTI-DEMENSIONAL MATRIX MULTIPLICATION}

The embodiments relate to a matrix multiplication method, an arithmetic device, and a program, and specifically, may be applied to a compressed multidimensional matrix multiplication method, an arithmetic device, and a program.

Recently, interest and utilization in the fields of artificial intelligence and deep learning are increasing. Accordingly, interest and utilization of deep neural networks for deep learning are increasing.

In order to learn and infer a deep neural network, a matrix multiplication operation of multiplying an input vector, matrix, or tensor with a weight matrix or tensor is required. This matrix multiplication operation is an important factor determining the operation speed as the depth and size of the deep neural network increase, and includes a quantization technique of each value in the matrix.

However, it is difficult to effectively process a multiplication operation on a quantized matrix using the existing sparse matrix technique, and the matrix in deep learning is not effective in terms of data storage space or amount of computation because it is often not a sparse matrix.

In addition, matrix multiplication operation using a matrix or tensor converted into a bit vector requires as many bit vectors as the number of distinct values present in data, so matrix values in deep learning with various values There is a problem that a large number of bit vectors are required to express . Furthermore, even if the compressed bit vector is compressed and used, there is a problem in that the multiplication operation cannot be performed on the bit vector during the matrix multiplication operation, and the process must be performed after decompression.

Accordingly, there is a need for a matrix multiplication method capable of directly applying the compressed data representation to a matrix multiplication operation.

A multidimensional matrix multiplication method according to embodiments may include performing quantization on a plurality of input matrices or tensors and outputting them as a plurality of binarized matrices or tensors; compressing a plurality of binarized matrices or tensors and converting them into a plurality of compressed bit vectors; calculating a bit at a specific position on a plurality of compressed bit vectors; outputting a result matrix by performing multiplication between a plurality of compressed bit vectors; and transforming the result matrix according to the position of the specific bit on the compressed bit vector; may include.

In addition, the converting into a plurality of compressed bit vectors may include: transforming a plurality of binarized matrices or tensors into a bit vector by unfolding them in one dimension; and performing compression of the converted bit vector by writing the value of the bit included in the converted bit vector. may include.

In addition, the value of the compressed bit vector may include whether the vector is compressed (0 or 1), the values 0 and 1, and the number of times 0 and 1 are repeated.

In addition, the plurality of input matrices include a first matrix and a second matrix to be subjected to matrix multiplication, and the outputting of the result matrix includes at least one of the number of columns of the first matrix or the number of rows of the second matrix. grouping bits included in the compressed bit vector by the number of logical blocks in units of logical blocks; performing a preset operation on the grouped bits; calculating the number of 1's included in a new bit vector obtained through a preset operation; and outputting a result matrix by setting the number of 1s as positions; may further include.

Also, the preset operation may be an exclusive nor (XNOR) operation and a bitcount operation.

In addition, the bits grouped in logical block units include a first group and a second group, and the step of outputting the result matrix includes, as the result matrix is output for the first group, the result matrix for the second group outputting; may include.

In addition, when the result matrix is output for all bits grouped in logical block units, circularly shifting a bit vector included in the compressed bit vector to the left by the size of the logical block unit; grouping bits included in a compressed bit vector by at least one of the number of columns of the first matrix and the number of rows of the second matrix in units of blocks; performing a preset operation on the grouped bits; calculating the number of 1's included in a new bit vector obtained through a preset operation; and outputting a result matrix by setting the number of 1s to positions; may further include.

A multidimensional matrix multiplication operation apparatus according to embodiments includes: a binarization module configured to output a plurality of binarized matrices by performing quantization on a plurality of input matrices; a bit vector compression module for converting a plurality of binarized matrices into a plurality of compressed bit vectors; a cursor module for calculating a position of a specific bit on a plurality of compressed bit vectors; and a multiplication operation module for performing multiplication of a plurality of compressed bit vectors to output a result matrix; may include

In addition, the bit vector compression module unfolds a plurality of binarized matrices or tensors, converts them into bit vectors, and records the values of bits included in the converted bit vectors to perform compression of the converted bit vectors. have.

In addition, the value of the compressed bit vector may include whether the vector is compressed (0 or 1), the values 0 and 1, and the number of times 0 and 1 are repeated.

In addition, the plurality of input matrices include a first matrix and a second matrix, and the multiplication operation module is included in the compressed bit vector by at least one of the number of columns of the first matrix or the number of rows of the second matrix. Grouping bits in logical block units, performing a preset operation on the grouped bits, calculating the number of 1s included in a new bit vector obtained through the preset operation, and setting the number of 1s as positions The result matrix may be output.

Also, the preset operation may be an exclusive nor (XNOR) operation and a bitcount operation.

In addition, the bits grouped in logical block units include a first group and a second group, and the multiplication operation module may output the result matrix to the second group as the result matrix is output to the first group. have.

In addition, when the multiplication operation module outputs a result matrix for all of the bits grouped in logical block units, the multiplication operation module circulates the bit vectors included in the compressed bit vectors by the size of the logical block units. Shifting, grouping bits included in the compressed bit vector by at least one of the number of columns of the first matrix or the number of rows of the second matrix in logical block units, and a preset operation on the grouped bits , calculates the number of 1s included in a new bit vector obtained through a preset operation, sets the number of 1s as positions, and outputs a result matrix.

A storage medium storing a multidimensional matrix multiplication program according to embodiments may include performing quantization on a plurality of input matrices or tensors and outputting them as a plurality of binarized matrices; transform the plurality of binarized matrices into a plurality of compressed bit vectors; calculating a bit value of a specific position on a plurality of compressed bit vectors; performing multiplication between a plurality of compressed bit vectors to output a result matrix; Corresponding to the position of a specific bit on the compressed bit vector, the result matrix may be transformed.

A storage medium for storing a compressed multidimensional matrix multiplication method, an arithmetic device, and a program according to the embodiments is a deep neural network model learning and inference process in deep learning through improvement of multiplication operation processing performance for large-scale multidimensional matrices and tensors. It can improve efficiency in both processing speed and storage space.

The multi-dimensional matrix multiplication method according to the embodiments, the computing device, and the storage medium for storing the program can efficiently process the deep learning model in embedded or mobile devices that do not have a high-performance computing device.

The multidimensional matrix multiplication method, arithmetic device, and storage medium for storing a program according to embodiments may greatly reduce the size of an input matrix to be calculated through a compressed bit vector.

The multi-dimensional matrix multiplication method, the calculation device, and the storage medium for storing the program according to the embodiments replace the matrix multiplication operation with a bit string operation on a compressed bit vector, thereby greatly reducing the calculation time in a low-performance device.

The multidimensional matrix multiplication method according to the embodiments, the arithmetic device and the storage medium for storing the program can efficiently perform inference through a deep learning model in a low-performance device.

The multi-dimensional matrix multiplication method, the calculation device, and the storage medium for storing the program according to the embodiments can more effectively perform the matrix multiplication operation in the lightweight neural network model through model quantization.

1 illustrates a structural diagram of a neural network according to embodiments.
2 illustrates matrix multiplication according to embodiments.
3 illustrates an example of performing matrix multiplication using a bit vector according to embodiments.
4 is a flowchart of a multidimensional matrix multiplication method according to embodiments.
FIG. 5 is a diagram schematically illustrating that binarizing an input matrix and matrix-multiplying it according to embodiments can approximate a result of matrix-multiplying existing input matrices as they are.
6 is a diagram schematically illustrating compression of a bit vector generated by unraveling a matrix and a tensor according to embodiments.
7 is a diagram schematically illustrating that a position of a specific bit is calculated by a cursor according to embodiments.
8 is a diagram schematically illustrating performing matrix multiplication through a compressed bit vector according to embodiments.
9 is a diagram schematically illustrating a configuration of a multidimensional matrix multiplication operation apparatus according to embodiments.

Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. In describing the embodiments, if it is determined that a detailed description of a related known technology may obscure the gist of the embodiment, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments, and should not be construed as limiting the technical idea by the accompanying drawings.

It is also understood that when an element, such as a layer, region, or substrate, is referred to as being “on” another component, it may be directly on the other element or intervening elements in between. There will be.

Multi-dimensional matrix multiplication described through embodiments can be used for both deep learning-based services for low-performance devices with computational performance, memory, and battery constraints such as mobile and embedded devices that need to introduce a deep learning model, Those skilled in the art will readily appreciate that even a new product form developed later can be applied to an arithmetic method or apparatus to which matrix multiplication is applied.

1 illustrates a structural diagram of a neural network according to embodiments.

1 , a neural network 100 according to embodiments may include an input layer 101 , a plurality of hidden layers 102 , and an output layer 103 . The neural network 100 according to the embodiments includes more layers from which valid information can be extracted, and thus can effectively process complex data sets. 1 illustrates one input layer 101 , three hidden layers 102 , and one output layer 103 , but the present invention is not limited thereto. It can contain more layers. Furthermore, the neural network 100 according to the embodiments may include layers having various structures different from those shown in FIG. 1 .

Each of the layers 101 , 102 , and 103 included in the neural network 100 according to embodiments is a neuron, a processing element (PE), a unit, or a plurality known as similar terms. may include artificial nodes of For example, as shown in FIG. 1 , the input layer 101 , the plurality of hidden layers 102 , and the output layer 103 may each include six nodes. However, the present invention is not limited thereto, and each of the layers 101 , 102 , and 103 may include a variable number of nodes.

Nodes included in each of the layers 101 , 102 , and 103 according to the embodiments may be connected to each other to transmit an operation result of the node to a node of a next layer. For example, one node may perform an operation by receiving data from other nodes of a previous layer, and may transmit an operation result to other nodes of a next layer. Each of the nodes according to the embodiments may determine its own output value based on activation functions and a weight value of output values received from nodes included in the previous layer.

In order to calculate the neural network 100 according to embodiments, matrix multiplication in which an input vector is multiplied by a weight may be performed. At this time, matrix multiplication is applied to all inputs of each layer, and as it is iteratively performed for each input vector, as the depth and size of the neural network increase, it can mean the most important basic operator that determines the overall operation speed. .

2 illustrates matrix multiplication according to embodiments.

2 illustrates an example of performing matrix multiplication using a compressed sparse matrix according to embodiments.

A sparse matrix according to embodiments may refer to a matrix in which there are many positions of 0 in the matrix and only a few non-zero values are present. When the sparse matrix is stored as it is, there is a problem of large space consumption due to a value of 0. Accordingly, matrix multiplication according to embodiments may include a compression method for indicating a non-zero matrix position.

FIG. 2A illustrates a CSR scheme as matrix multiplication according to embodiments.

It may be matrix multiplication using a Compressed Sparse Row (CSR) method according to embodiments. For example, as shown in (a) of FIG. 2 , in the existing matrix 210 , non-zero values are assigned to the data vector 211 in the order from left to right from the top row. can be saved That is, the data vector 211 may store a value of 8257129. The indices vector 212 may sequentially store position values of columns in which non-zero values are located in the existing matrix 210 . For example, 0, 2, 2, 2, 3, 4, 3, which are columns in which non-zero values 8, 2, 5, 7, 1, 2, and 9 included in the existing matrix 210 are located, are ordered in each order. You can save it as 0222343. The index pointer vector 213 may record the order of data as a range. For example, since the data of the first row exists within the range of [0:2], it can be seen that the number of data is two, and it can be seen that there are the first two, that is, 8 and 2 in the data vector 211 . can be cut The next row can be written as [3:3] because there is no non-zero value in the next row.

FIG. 2B illustrates a CSC method as matrix multiplication according to embodiments.

It may be matrix multiplication using a Compressed Sparse Column (CSC) method according to embodiments. In this case, the CSC method proceeds in the same manner as the CSR method, but data may be recorded centered on columns rather than rows. For example, as shown in FIG. 2B , in the existing matrix 220 , non-zero values can be stored in the data vector 221 in the order from top to bottom from the leftmost column. have. That is, the data vector 221 may store a value of 8257192. The exponential vector 222 may sequentially store position values of rows in which non-zero values are located in the existing matrix 220 . For example, rows in which non-zero values included in the existing matrix 220 are located may be sequentially stored as 0014464. The index pointer vector 223 may record the order of data as a range, and may have a value of, for example, 011467.

However, an expression method using a compressed sparse matrix is an effective method for sparse matrices, but matrices in deep learning are often not sparse matrices, so there is a problem in that they are not effective in terms of data storage space or computational amount. Accordingly, a calculation method capable of expressing various values in a neural network is required.

3 illustrates an example of performing matrix multiplication using a bit vector according to embodiments.

3 illustrates an example of performing matrix multiplication using a compressed bit vector according to embodiments.

A matrix multiplication method using a compressed bit vector according to embodiments may include converting data 301 into a bit vector 302 and converting the bit vector 302 into a compressed bit vector 303 .

The bit vector 302 according to the embodiments generates bit vectors as much as a unique value of the data 301 and stores the position of a specific value in the data by expressing it as 1 in the bit vector expressing the corresponding value. can For example, in the case of 1, which is the first value included in the data 301 , it may be expressed as the first 1 of the bit vector 302 for 1. In addition, in the case of 3, which is the second value included in the data 301 , it can be expressed as 1 in the second position of the bit vector 302 for 3 .

The compressed bit vector 303 according to embodiments may use a Run Length Encoding (RLE) method as a method of compressing repetitive 0s or 1s to express a small value. Specifically, it is possible to record the length of a run for the same value that appears repeatedly, for example, as shown in FIG. 3 , it can record that 8*0 occurs twice.

However, the expression method using compressed bit vectors has a problem in that a large number of bit vectors are required to express matrix values in deep learning having various values as the number of bit vectors must exist as many as the number of unique values present in the data. . In addition, the expression method using the compressed bit vector has a problem in that the operation cannot be performed immediately and the compression must be decompressed. Accordingly, there is a need for an operation method capable of more effectively performing a matrix multiplication operation in a neural network.

4 is a flowchart of a multidimensional matrix multiplication method according to embodiments.

As shown in FIG. 4 , the multidimensional matrix multiplication method according to the embodiments may include outputting a plurality of input matrices as a plurality of binarized matrices ( S401 ).

A plurality of input matrices according to embodiments may include input matrices for performing matrix multiplication as a matrix to be calculated. The plurality of input matrices may include a multidimensional matrix. The plurality of input matrices may have, for example, a three-dimensional matrix. A plurality of input matrices may be output as a binarized matrix through quantization. At this time, quantization is a value configured to have a smaller bit length for element values of the original matrix. It may be an operation to express with a small bit length. The plurality of input matrices may include a first matrix and a second matrix.

According to embodiments, the binarized matrix may be a matrix in which values included in the input matrix are binarized through quantization or are composed of values decomposable into binarized values. The binarized matrix has the same appearance as the input matrix, but values included in the binarized matrix may have binarized values.

As shown in FIG. 1 , the multidimensional matrix multiplication method according to embodiments may include compressing a plurality of binarized matrices and converting the plurality of binarized matrices into a plurality of compressed bit vectors ( S402 ).

According to embodiments, the binarized matrix may be converted into a one-dimensional bit vector in an unfolded state. The converted bit vector may be compressed through a Run Length Encoding (RLE) method. Specifically, the converted bit vector may be compressed by writing the value of the bit included in the converted bit vector.

A bit vector compressed according to embodiments may include 0, 1, and the number of times 0 or 1 is repeated as bits. Specifically, the value of the compressed bit vector may include whether the vector is compressed (0 or 1), the values 0 and 1, and the number of times 0 and 1 are repeated. Accordingly, the multidimensional matrix multiplication method according to the embodiments may greatly reduce the size of the input matrix through a compressed bit vector.

As shown in FIG. 4 , the multidimensional matrix multiplication method according to embodiments may include calculating a bit at a specific position on a plurality of compressed bit vectors ( S403 ).

In the compressed bit vector according to embodiments, the position of a specific bit existing in the compressed bit vector by a cursor or the bit value of the specific position may be determined. In this case, the cursor is a virtual cursor, and may be for reducing position information of a specific bit with respect to a compressed bit vector to position information of a specific bit with respect to an uncompressed bit vector. In this case, the cursor may acquire a position in a currently compressed word unit and a visited position in a compressed form corresponding to the position in order to acquire position information of a specific bit with respect to the uncompressed bit vector. .

In the multidimensional matrix multiplication method according to the embodiments, when 0 is included in a size unit in which the size of a bit is divided through a cursor, the processing speed may be reduced by skipping an area including 0 when irradiating it. . Specifically, the multidimensional matrix multiplication method according to the embodiments may skip a bit visit unit or move the cursor to a position of a specific bit until a bit including 1 comes out through the cursor.

As shown in FIG. 4 , the multidimensional matrix multiplication method according to embodiments may include performing multiplication between a plurality of compressed bit vectors to output a result matrix ( S404 ). In this case, in order to perform multiplication between compressed bit vectors, an XNOR operator and a bitcount operation for calculating the number of bits having a value of 1 may be included.

According to embodiments, the compressed bit vector may be logically grouped for calculation. Specifically, the bits included in the compressed bit vector may be grouping in which groups are divided into logical block units. The compressed bit vector may be grouped in logical block units including the number of bits corresponding to the number of columns of the first matrix, which is the input matrix. Also, the compressed bit vector may be grouped in logical block units including the number of bits corresponding to the number of rows of the second matrix, which is the input matrix.

According to embodiments, it may be determined whether grouped bits are matched or mismatched with each other by a preset operation. Specifically, it may be determined whether the plurality of grouped bits, which are the target of the operation, match each other or mismatch many times through a preset operation. In this case, the preset operation may be an exclusive NOR (XNOR). That is, the number of 1s may be calculated for the grouped bits through a preset operation, for example, an XNOR operation using the bitcount function.

In the result matrix output according to embodiments, a position of a bit may be set based on the calculated number of ones. Specifically, in the output result matrix, a position at which a value of each bit is input may be set based on the number of 1's.

That is, the multidimensional matrix multiplication method according to the embodiments may output a result matrix by performing multiplication between compressed bit vectors in an uncompressed state, that is, in a state in which the bit vectors are compressed.

As shown in FIG. 4 , the multidimensional matrix multiplication method according to embodiments may include transforming the result matrix corresponding to the position of a specific bit on a compressed bit vector ( S405 ).

A result matrix output according to embodiments may be converted into an uncompressed result matrix based on position information of a specific bit with respect to an uncompressed bit vector obtained by a cursor.

In this case, according to embodiments, it may be determined whether the logically grouped bits match or mismatch each other by a preset operation. Specifically, it may be determined whether the plurality of grouped bits, which are the target of the operation, match each other or mismatch many times through a preset operation. In this case, the preset operation may be XNOR. That is, the number of 1s may be calculated for the grouped bits through a preset operation, for example, an XNOR operation using the bitcount function. Based on the calculated number of 1s, the position of the bit may be set. Specifically, in the result matrix to be output, the position at which the value of each bit is input may be set based on the number of 1's.

In the multidimensional matrix multiplication method according to the embodiments, when the operation on the bits belonging to the first group of the compressed bit vector is finished, the operation may be repeatedly performed on the bits belonging to the second group. In this case, the second group may perform step s403.

In the multidimensional matrix multiplication method according to the embodiments, when all of the steps s401 to s403 are performed with respect to the entire block unit (including the first group and the second group) divided into groups according to the embodiments, the block unit By cyclically shifting the bit vector to the left by the size, the step s404 may be performed.

Hereinafter, outputting the input matrix as a binarized matrix will be described in detail.

FIG. 5 is a diagram schematically illustrating that binarizing an input matrix and multiplying it by a matrix according to embodiments can approximate a result of matrix multiplication of existing input matrices.

As shown in FIG. 5 , the input matrix 510 (eg, the input layer described in FIG. 1 , the data described in FIGS. 2 to 3 , and the input matrix described in FIG. 4 ) is a plurality of multidimensional calculation objects. May contain matrices or tensors. In FIG. 5 , the input matrix 510 is illustrated in three dimensions, but the present invention is not limited thereto, and the supported dimension may be any dimension. The input matrix 510 may include, for example, a first matrix (I) and a second matrix (W).

As shown in FIG. 5 , values included in the input matrix 510 may be binarized and output as a binarized matrix 520 (eg, the binarized matrix described in FIG. 4 ). Specifically, values included in the input matrix 510 may be binarized by performing quantization or output as a binarized matrix 520 composed of values decomposable into binarized values. In this case, the identification number 521 may be a matrix (K) or a constant (α) for returning to a value before decomposition when the input matrix can be decomposed into a binarized value. For example, the value 0.2 of the first matrix (I) included in the input matrix 510 becomes 1 while being binarized, and the value -5 of the second matrix (I) included in the input matrix 510 is binarized while - can be 1. In this case, the binarized matrix 520 may include bits that are configured to have a smaller bit length than the input matrix 510 and may have the same or similar value to the input matrix 510 . The binarized matrix 520 may be converted into a bit vector.

Hereinafter, the compression of the binarized matrix converted into a bit vector and conversion into a compressed bit vector will be described in detail.

6 is a diagram schematically illustrating compression of a bit vector generated by unraveling a matrix and a tensor according to embodiments.

The multidimensional matrix multiplication method (eg, the matrix multiplication described in FIGS. 1 to 3 , the multidimensional matrix multiplication method described in FIGS. 4 to 5 ) according to the embodiments is a binarized matrix (eg, FIGS. 4 to 5 ) It may include a step (eg, s402 described in FIG. 4 ) representing the binarized matrix described in ) as a compressed bit vector (eg, the compressed bit vector described in FIG. 4 ).

6(a) shows a bit vector in which a binarized matrix and a tensor are unfolded. An unfolded bit vector consists of, for example, 8,000 bits, where the bit vector can only have a value of 0 or 1. For convenience of description, hereinafter, it is assumed that one word in a computing device including a computing device consists of 32 bits, but the present invention is not limited thereto.

As shown in (b) of FIG. 6 , since one word is 32 bits, grouping can be performed by 31 bits. For example, 8000 bits are grouped into 258 groups with respect to 7998 bits, and 2 bits may remain. In this case, the 258 groups may be divided into a group including 1 and a group not including 1. For example, 2 groups may include 1, and 256 groups may be groups that do not include 1. In this case, as shown in (b) of FIG. 6 , a group including 1 and a group not including 1 may be separated and expressed as 31 bits and 256 * 31 bits.

As shown in (c) of FIG. 6 , compression may not be performed on a group including 1. In this case, the first bit of the group may be expressed as 0.

As shown in (c) of FIG. 6 , compression may be performed on a group that does not include 1 . In this case, the first bit of the group may be expressed as 1. In this case, the number of times the group is repeated 256 (Run-length) may be recorded in the remaining bits, for example, 100000000 may be recorded.

That is, as shown in (c) of FIG. 6 , by performing compression according to whether 1 is included in each group, a compressed bit vector may be output.

Hereinafter, calculating the position of a specific bit on the compressed bit vector will be described in detail.

7 is a diagram schematically illustrating that a position of a specific bit is calculated by a cursor according to embodiments.

A multidimensional matrix multiplication method (eg, the matrix multiplication described with reference to FIGS. 1 to 3 , and the multidimensional matrix multiplication method described with reference to FIGS. 4 to 6 ) according to embodiments is a cursor (eg, the cursor or virtual method described with reference to FIG. 4 ) It may include a step of obtaining the position information of a specific bit (eg, s403 described in FIG. 4 ) through a cursor), and specifically, a compressed bit vector (eg, the compression described in FIGS. 4 and 6 ). bit vector), information for reducing the position information of a specific bit to position information in an uncompressed state can be obtained.

A cursor according to embodiments may skip a bit-by-bit visit until the next 1 bit is displayed, as shown in FIG. 7 a ). For example, by skipping a unit bit including only 0 among unit bits, it is possible to move from a unit bit including 1 to a unit bit including 1.

As shown in b) of FIG. 7 , the cursor according to embodiments may move the cursor to a position of a specific bit. For example, the cursor may move to the position of a specific bit with a '1'.

Hereinafter, matrix multiplication using a compressed bit vector will be described in detail.

8 is a diagram schematically illustrating performing matrix multiplication through a compressed bit vector according to embodiments.

The multidimensional matrix multiplication method (eg, the matrix multiplication described in FIGS. 1 to 3 , the multidimensional matrix multiplication method described in FIGS. 4 to 6 ) according to the embodiments is the result matrix 830 (eg, in FIG. 1 ). outputting the described output layer and the result matrix described with reference to FIG. 4 (eg, s404 to s405 described with reference to FIG. 4 ). Specifically, in the multidimensional matrix multiplication method, the input matrix 810 (eg, the input layer described in FIG. 1 , the data described in FIGS. 2 to 3 , and the input matrix described in FIGS. 4 to 5 ) is converted into an unfolded vector. It is represented by 820 (eg, the unfolded matrix described with reference to FIG. 6 ), and an unfolded vector may be output as the result matrix 830 .

The input matrix 810 according to embodiments may include an input matrix I and a weight matrix W. The input matrix I and the weight matrix W may be transformed into a matrix having a real value of 0 or 1 by the quantization technique described with reference to FIGS. 4 to 5 . Values of 1 to 15 of the input matrices 810 shown in FIG. 8 are numerically indicating positions of values of the input matrices 810 in an easy to understand manner. That is, the input matrix 810 may be transformed into a binarized matrix (eg, the binarized matrix described with reference to FIGS. 4 to 6 ) through a quantization technique.

The binarized matrix according to embodiments may be converted into an unfolded vector 820 through the unfolding process described with reference to FIGS. 4 and 6 . Through the unfolding process, the vector values of the input vector I and the vector values of the weight vector W can be arranged on unfold(I) and unfold(W), which are the unfolded vectors. In this case, numbers corresponding to unfold(I) and unfold(W), for example, 1 included in unfold(I) and 1 corresponding to unfold(W) may represent values that should be multiplied by each other.

In this case, in response to the loss of distinction between the same row or the same column in the unfolding process according to the embodiments, the multidimensional matrix multiplication method according to the embodiments includes bits as many as the number of columns of the input matrix I or the number of rows of the weighting matrix W. Grouping may be performed by grouping them into logical block units, and accordingly, a grouped bit (eg, the grouped bit described with reference to FIG. 4 ) may be formed.

In this case, the multidimensional matrix multiplication method according to the embodiments may obtain a result value by performing XNOR and/or bitcount within the grouped interval, for example, Bitcount(XNOR(I,W)). can do.

As shown in FIG. 8 , after outputting the result matrix 831 with respect to the unfolded matrix 821 , for the next interval in the vector through a left cyclic shift (eg, the cyclic shift described in FIG. 4 ) The operation can be repeated. That is, an operation may be performed on the matrix 822 expressed as a bit vector, and thus the result matrix 832 may be output.

In this case, the result matrix 830 may record the result values on ①, ②, ③, ④, ⑤, and ⑥ in order. Through this, efficient operation between multidimensional matrices can be performed by minimizing the number of operations applied in units of bits.

In FIG. 8, for convenience of explanation, vectors are calculated through XNOR operation in an uncompressed state to indicate the positions of matrix elements, but in actual processing, as shown in FIGS. 6 to 7, compressed bit vectors The operation shown in FIG. 8 may be performed with respect to .

9 is a diagram schematically illustrating a configuration of a multidimensional matrix multiplication operation apparatus according to embodiments.

The multidimensional matrix multiplication operation apparatus 900 according to embodiments may include a binarization module 910 , a bit vector compression module 920 , a cursor module 930 , and a multiplication operation module 940 .

The binarization module 910 according to embodiments may include a plurality of input matrices (eg, the input layer described in FIG. 1 , the data described in FIGS. 2 to 3 , and the input matrix described in FIGS. 4 to 5 and 8 ). A plurality of binarized matrices (eg, the binarized matrices described with reference to FIGS. 4 to 6 and 8 ) may be output by performing quantization. The binarization module 910 may, for example, perform s401 described with reference to FIG. 4 .

The bit vector compression module 920 according to embodiments may convert a plurality of binarized matrices into a plurality of compressed bit vectors (eg, the compressed bit vectors described with reference to FIGS. 4 and 6 to 7 ). have. The bit vector compression module 920 may, for example, perform s402 described with reference to FIG. 4 . Specifically, the bit vector compression module 920 may unfold a plurality of binarized matrices to convert them into bit vectors, and perform compression of the converted bit vectors by recording the values of bits included in the converted bit vectors. . In this case, the value of the compressed bit vector may include 0, 1, and the number of times 0 and 1 are repeated.

The cursor module 930 according to embodiments may calculate a position of a specific bit or calculate a bit value of a specific position on a plurality of compressed bit vectors. The cursor module 930 may acquire a specific bit (eg, a position of 1) using a cursor (eg, the cursor or virtual cursor described with reference to FIGS. 4 to 5 ). . can run

The multiplication operation module 940 according to embodiments may perform multiplication of a plurality of compressed bit vectors to output a result matrix (eg, the result matrix described with reference to FIGS. 4 and 8 ). The multiplication operation module 940 may, for example, perform operations s404 and s405 described with reference to FIG. 4 . Specifically, the multiplication operation module 940 may group the bits included in the compressed bit vector by at least one of the number of columns and the number of rows included in the plurality of input matrices in block units. The multiplication operation module 940 may perform a preset operation on the grouped bits. In this case, the preset operation may be an exclusive NOR (XNOR). That is, the multiplication operation module 940 may calculate the number of 1s included in the grouped bits through a preset operation, for example, an XNOR operation using a bitcount function. The multiplication operation module 940 may output a result matrix by setting the number of 1s to positions. When the result matrix is output for the first group, the multiplication operation module 940 may perform multiplication between compressed bit vectors for the next group. When multiplication is performed on all grouped bits, the multiplication operation module 940 cyclically shifts a bit vector included in a bit vector compressed by a block unit size (eg, as described in FIGS. 4 and 8 ). cyclic shift) to perform multiplication of compressed bit vectors.

A storage device including a multidimensional matrix multiplication method, an arithmetic device, and a program according to the embodiments is a low-performance PC, embedded, and mobile device that does not have a high-performance computing device such as a GPU through a matrix multiplication speeding technique using a compressed bit vector. The learning model can be processed efficiently.

Terms such as first, second, etc. may be used to describe various components of the embodiments. However, the interpretation of various components according to the embodiments should not be limited by the above terms. These terms are only used to distinguish one component from another. it is only For example, the first user input signal may be referred to as a second user input signal. Similarly, the second user input signal may be referred to as a first user input signal. Use of these terms should be interpreted as not departing from the scope of the various embodiments. Although both the first user input signal and the second user input signal are user input signals, they do not mean the same user input signals unless the context clearly indicates otherwise.

The terminology used to describe the embodiments is used for the purpose of describing specific embodiments, and is not intended to limit the embodiments. As used in the description of the embodiments and in the claims, the singular is intended to include the plural unless the context clearly dictates otherwise. and/or expressions are used in their sense to include all possible combinations between terms. The expression to include describes the presence of features, numbers, steps, elements, and/or components, and does not mean to exclude additional features, numbers, steps, elements, and/or components. . Conditional expressions such as when, when, etc. used to describe the embodiments are not limited to only optional cases. When a specific condition is satisfied, a related action is performed in response to the specific condition, or a related definition is intended to be interpreted.

The above description is merely illustrative of the technical idea, and various modifications and variations will be possible without departing from the essential characteristics of the embodiments by those of ordinary skill in the art to which the embodiments belong.

Therefore, the embodiments disclosed above are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit is not limited by the embodiments of the present invention.

The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

810: input matrix
820: Unfolded Vector
830: result matrix

Claims (15)

performing quantization on a plurality of input matrices and outputting them as a plurality of binarized matrices;
converting the plurality of binarized matrices into a plurality of compressed bit vectors;
calculating a position of a specific bit on the plurality of compressed bit vectors;
outputting a result matrix by performing multiplication between the plurality of compressed bit vectors; and
transforming the result matrix according to the position of the specific bit on the compressed bit vector; containing,
Multidimensional matrix multiplication method. The method of claim 1,
The step of converting the plurality of compressed bit vectors comprises:
converting the plurality of binarized matrices into bit vectors by unfolding them;
performing compression of the converted bit vector by writing a bit value included in the converted bit vector; containing,
Multidimensional matrix multiplication method. 3. The method of claim 2,
The value of the bit vector includes 0, 1, and the number of times 0 and 1 are repeated.
Multidimensional matrix multiplication method. The method of claim 1,
The plurality of input matrices includes a first matrix and a second matrix,
Outputting the result matrix comprises:
grouping bits included in the compressed bit vector into logical blocks by at least one of the number of columns of the first matrix and the number of rows of the second matrix;
performing a preset operation on the grouped bits;
calculating the number of 1's included in the new bit vector obtained through the preset operation; and
outputting the result matrix by setting the number of 1s as positions; further comprising
Multidimensional matrix multiplication method. 5. The method of claim 4,
The preset operation is an XNOR (exclusive nor) operation and a Bitcount operation,
Multidimensional matrix multiplication method. 5. The method of claim 4,
The bits grouped in units of logical blocks include a first group and a second group,
Outputting the result matrix comprises:
outputting a result matrix for the second group as the result matrix is output for the first group; containing
Multidimensional matrix multiplication method. 7. The method of claim 6,
When the result matrix is output for all bits grouped in the logical block unit,
circularly shifting a bit vector included in the compressed bit vector by the size of the logical block unit;
grouping bits included in the compressed bit vector in logical block units by at least one of the number of columns of the first matrix and the number of rows of the second matrix;
performing a preset operation on the grouped bits;
calculating the number of 1's included in the new bit vector obtained through the preset operation; and
outputting the result matrix by setting the number of 1s as positions; further comprising
Multidimensional matrix multiplication method. a binarization module for outputting a plurality of binarized matrices by performing quantization on a plurality of input matrices;
a bit vector compression module converting the plurality of binarized matrices into a plurality of compressed bit vectors;
a cursor module for calculating a bit at a specific position on the plurality of compressed bit vectors; and
a multiplication operation module for performing multiplication of the plurality of compressed bit vectors and outputting a result matrix;
containing
Multidimensional matrix multiplication operation unit. 9. The method of claim 8,
The bit vector compression module,
unfolding the plurality of binarized matrices and transforming them into bit vectors,
performing compression of the converted bit vector by writing the value of the bit included in the converted bit vector,
Multidimensional matrix multiplication operation unit. 10. The method of claim 9,
The value of the bit vector includes 0, 1, and the number of times 0 and 1 are repeated.
Multidimensional matrix multiplication operation unit. 9. The method of claim 8,
The plurality of input matrices includes a first matrix and a second matrix,
The multiplication operation module is
Grouping bits included in the compressed bit vector by at least one of the number of columns of the first matrix or the number of rows of the second matrix in logical block units,
performing a preset operation on the grouped bits,
Calculate the number of 1s included in the new bit vector obtained through the preset operation,
Outputting the result matrix by setting the number of 1s to positions,
Multidimensional matrix multiplication operation unit. 12. The method of claim 11,
The preset operation is an XNOR operation and a Bitcount operation,
Multidimensional matrix multiplication operation unit. 12. The method of claim 11,
The bits grouped in units of logical blocks include a first group and a second group,
The multiplication operation module is
outputting a result matrix for the second group as the result matrix is output for the first group,
Multidimensional matrix multiplication operation unit. 14. The method of claim 13,
When the multiplication operation module outputs a result matrix for all of the bits grouped in the logical block unit,
The multiplication operation module is
Cycically shifting a bit vector included in the compressed bit vector by the size of the logical block unit,
Grouping bits included in the compressed bit vector by at least one of the number of columns of the first matrix or the number of rows of the second matrix in logical block units,
performing a preset operation on the grouped bits,
Calculate the number of 1s included in the new bit vector obtained through the preset operation,
Outputting the result matrix by setting the number of 1s as positions,
Multidimensional matrix multiplication operation unit. performing quantization on a plurality of input matrices and outputting them as a plurality of binarized matrices;
transform the plurality of binarized matrices into a plurality of compressed bit vectors;
calculating a position of a specific bit on the plurality of compressed bit vectors;
performing multiplication between the plurality of compressed bit vectors to output a result matrix;
transforming the result matrix, corresponding to the position of the specific bit on the compressed bit vector;
A storage medium storing a multidimensional matrix multiplication program.

Images