KR102572429B1 - Method, apparatus and storage for storing a program for multi-demensional matrix multiplication - Google Patents

Abstract

Embodiments relate to a multidimensional matrix multiplication method, an arithmetic device, and a program for improving multidimensional matrix multiplication processing performance, and a resulting matrix may be derived through multiplication between compressed bit vectors.

Description

Compressed multidimensional matrix multiplication method, arithmetic device, and storage medium for storing programs

Embodiments relate to a matrix multiplication method, arithmetic device, and program, and may be specifically applied to a compressed multidimensional matrix multiplication method, arithmetic device, and program.

Recently, interest in and utilization of artificial intelligence and deep learning fields are increasing. Accordingly, interest in and utilization of deep neural networks for deep learning are increasing.

For learning and inference of a deep neural network, a matrix multiplication operation that multiplies an input vector, matrix, or tensor with a weight matrix or tensor is required. This matrix multiplication operation is an important factor that determines the operation speed as the depth and size of the deep neural network increase, and includes a quantization technology of each value in the matrix.

However, multiplication operations for quantized matrices are difficult to effectively process with existing sparse matrix techniques, and matrices in deep learning are often not sparse matrices, so there is a problem in that they are not effective in terms of data storage space or amount of computation.

In addition, the matrix multiplication operation by converting a matrix or tensor into a bit vector and using it requires that as many bit vectors as the number of distinct values exist in the data, matrix values in deep learning with various values There is a problem that requires a large number of bit vectors when expressing . Furthermore, even if the compressed bit vector is compressed and used, there is a problem in that the matrix multiplication operation cannot perform the multiplication operation on the bit vector and must be processed after decompression.

Therefore, a matrix multiplication method capable of directly applying the compressed data expression to a matrix multiplication operation is required.

A multidimensional matrix multiplication method according to embodiments may include performing quantization on a plurality of input matrices or tensors and outputting a plurality of binarized matrices or tensors; compressing a plurality of binarized matrices or tensors and converting them into a plurality of compressed bit vectors; calculating a bit at a specific position on a plurality of compressed bit vectors; outputting a resulting matrix by performing multiplication between a plurality of compressed bit vectors; and transforming a resulting matrix corresponding to the position of a specific bit on the compressed bit vector; can include

In addition, the step of converting into a plurality of compressed bit vectors may include converting a plurality of binarized matrices or tensors into one-dimensional bit vectors by unfolding them; and performing compression of the converted bit vector by recording values of bits included in the converted bit vector. can include

In addition, the value of the compressed bit vector may include whether the vector is compressed (0 or 1), values of 0 and 1, and the number of repetitions of 0 and 1.

In addition, the plurality of input matrices include first matrices and second matrices to be subjected to matrix multiplication, and the step of outputting the resulting matrix includes at least one of the number of columns of the first matrix and the number of rows of the second matrix grouping bits included in the compressed bit vector by the number of logical blocks; performing a preset operation on the grouped bits; calculating the number of 1s included in a new bit vector obtained through a predetermined operation; and outputting a resultant matrix by setting the number of 1s as a position; may further include.

Also, the preset operation may be an exclusive nor (XNOR) operation and a bitcount operation.

In addition, the bits grouped in logical block units include a first group and a second group, and outputting the result matrix, as the result matrix for the first group is output, outputs the result matrix for the second group. outputting; can include

In addition, when the result matrix is output for all of the bits grouped in logical block units, circularly shifting the bit vector included in the compressed bit vector to the left by the size of the logical block unit; grouping bits included in a bit vector compressed by at least one of the number of columns of the first matrix and the number of rows of the second matrix in block units; performing a preset operation on the grouped bits; calculating the number of 1s included in a new bit vector obtained through a predetermined operation; and outputting a resultant matrix by setting the number of 1s as positions; may further include.

A multidimensional matrix multiplication operation apparatus according to embodiments includes a binarization module that performs quantization on a plurality of input matrices and outputs a plurality of binarized matrices; a bit vector compression module for converting a plurality of binarized matrices into a plurality of compressed bit vectors; A cursor module that calculates a position of a specific bit on a plurality of compressed bit vectors; and a multiplication operation module for performing multiplication of a plurality of compressed bit vectors and outputting a resulting matrix. can include

In addition, the bit vector compression module converts a plurality of binarized matrices or tensors into bit vectors by unfolding them, and records the values of bits included in the converted bit vectors to compress the converted bit vectors. there is.

In addition, the value of the compressed bit vector may include whether the vector is compressed (0 or 1), values of 0 and 1, and the number of repetitions of 0 and 1.

In addition, the plurality of input matrices include a first matrix and a second matrix, and the multiplication operation module is included in the bit vector compressed by at least one of the number of columns of the first matrix and the number of rows of the second matrix Bits are grouped in logical block units, a preset operation is performed on the grouped bits, the number of 1s included in the new bit vector obtained through the preset operation is calculated, and the number of 1s is set as a position. The resulting matrix may be output.

Also, the preset operation may be an exclusive nor (XNOR) operation and a bitcount operation.

In addition, the bits grouped in logical block units include a first group and a second group, and the multiplication operation module may output a result matrix for the second group as the result matrix is output for the first group. there is.

In addition, when the multiplication operation module outputs a result matrix for all of the bits grouped in logical block units, the multiplication operation module cycles the bit vector included in the compressed bit vector by the size of the logical block unit. Shift, group the bits included in the bit vector compressed by at least one of the number of columns of the first matrix or the number of rows of the second matrix in logical block units, and perform a predetermined operation on the grouped bits , calculates the number of 1's included in the new bit vector obtained through a preset operation, sets the number of 1's as a position, and outputs the resulting matrix.

A storage medium storing a multidimensional matrix multiplication program according to embodiments performs quantization on a plurality of input matrices or tensors and outputs them as a plurality of binarized matrices; converting a plurality of binarized matrices into a plurality of compressed bit vectors; Calculate a bit value at a specific position on the plurality of compressed bit vectors; performing multiplication between a plurality of compressed bit vectors to output a resulting matrix; Corresponding to the location of a specific bit on the compressed bit vector, the resulting matrix may be transformed.

The storage medium for storing the compressed multidimensional matrix multiplication method, operation device, and program according to the embodiments improves the processing performance of multidimensional matrices and tensors for large-scale multidimensional matrices and tensors in the deep neural network model learning and reasoning process in deep learning. Efficiency can be improved in both processing speed and storage space.

The storage medium for storing the multidimensional matrix multiplication method, computing device, and program according to the embodiments can efficiently process a deep learning model in an embedded or mobile device that does not have a high-performance computing device.

A storage medium storing a multidimensional matrix multiplication method, an arithmetic device, and a program according to embodiments can significantly reduce the size of an input matrix to be calculated through a compressed bit vector.

The multidimensional matrix multiplication method, arithmetic unit, and storage medium storing the program according to the embodiments can significantly reduce calculation time in low-performance devices by replacing matrix multiplication with bit string operations on compressed bit vectors.

The storage medium for storing the multidimensional matrix multiplication method, the computing device, and the program according to the embodiments can efficiently perform inference through a deep learning model in a low-performance device.

A storage medium storing a multidimensional matrix multiplication method, an arithmetic unit, and a program according to embodiments can more effectively perform a matrix multiplication operation in a lightweight neural network model through model quantization.

1 shows a structural diagram of a neural network according to embodiments.
2 illustrates matrix multiplication according to embodiments.
3 illustrates an example of performing matrix multiplication using bit vectors according to embodiments.
4 is a flowchart of a multidimensional matrix multiplication method according to embodiments.
5 is a diagram schematically showing that binarization of an input matrix and matrix multiplication thereof according to embodiments can be approximated with a result of matrix multiplication of existing input matrices as they are.
6 is a diagram schematically illustrating compression of a bit vector generated by solving a matrix and a tensor according to embodiments.
7 is a diagram schematically illustrating that a position of a specific bit is calculated by a cursor according to embodiments.
Figure 8 is a schematic diagram of performing matrix multiplication over a compressed bit vector according to embodiments.
9 is a diagram schematically showing the configuration of a multidimensional matrix multiplication operation device according to embodiments.

Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings, but the same or similar elements are given the same reference numerals regardless of reference numerals, and overlapping descriptions thereof will be omitted. In describing the embodiments, if it is determined that a detailed description of related known technologies may obscure the gist of the embodiment, the detailed description will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments, and should not be construed as limiting the technical idea by the accompanying drawings.

It is also to be understood that when an element such as a layer, region or substrate is referred to as being “on” another element, it may be directly on the other element or intervening elements may exist therebetween. There will be.

The multidimensional matrix multiplication described through the embodiments can be used for all deep learning-based services for low-performance devices with computational performance, memory, and battery limitations such as mobile and embedded devices that need to introduce a deep learning model, A person skilled in the art will readily recognize that even a new product form to be developed later can be applied to an arithmetic method or device to which matrix multiplication is applied.

1 shows a structural diagram of a neural network according to embodiments.

As shown in FIG. 1 , a neural network 100 according to embodiments may include an input layer 101, a plurality of hidden layers 102, and an output layer 103. The neural network 100 according to the embodiments includes more layers from which valid information can be extracted, and thus can effectively process complex data sets. 1 shows one input layer 101, three hidden layers 102, and one output layer 103, but is not limited thereto, and the neural network 100 according to embodiments has fewer or It may contain more layers. Furthermore, the neural network 100 according to the embodiments may include layers having various structures different from those shown in FIG. 1 .

Each of the layers 101, 102, and 103 included in the neural network 100 according to the embodiments includes a plurality of layers known as a neuron, a processing element (PE), a unit, or similar terms. may include artificial nodes of For example, as shown in FIG. 1 , each of the input layer 101, the plurality of hidden layers 102, and the output layer 103 may include 6 nodes. However, it is not limited thereto, and each of the layers 101, 102, and 103 may include various numbers of nodes.

Nodes included in each of the layers 101, 102, and 103 according to the embodiments are connected to each other, and the operation result of the node can be transmitted to the node of the next layer. For example, one node may receive data from other nodes of a previous layer, perform calculations, and transfer the calculation result to other nodes of a next layer. Each of the nodes according to the embodiments may determine its own output value based on a weight value of output values received from nodes included in the previous layer and activation functions.

For calculation of the neural network 100 according to the embodiments, matrix multiplication for multiplying input vectors by weights may be performed. At this time, matrix multiplication is applied to all inputs of each layer, and as it is repeatedly performed for each input vector, it can mean the most important basic operator that determines the overall operation speed as the depth and size of the neural network increase. .

2 illustrates matrix multiplication according to embodiments.

2 illustrates an example of performing matrix multiplication using a compressed sparse matrix according to embodiments.

A sparse matrix according to embodiments may refer to a matrix in which there are many positions of 0 in the matrix and a small portion of non-zero values. When a sparse matrix is stored as it is, there is a problem in that space consumption due to a value of 0 is large. Accordingly, matrix multiplication according to embodiments may include a compression scheme for marking a non-zero matrix position.

2(a) shows a CSR method as matrix multiplication according to embodiments.

It may be matrix multiplication using a compressed sparse row (CSR) method according to embodiments. For example, as shown in (a) of FIG. 2, in the existing matrix 210, non-zero values are assigned to the data vector 211 from the top row in order from left to right. can be saved That is, the data vector 211 can store 8257129 values. The indices vector 212 may sequentially store position values of columns in which non-zero values are located in the existing matrix 210 . For example, 0, 2, 2, 2, 3, 4, 3, which are columns where non-zero values 8, 2, 5, 7, 1, 2, and 9 are located in the existing matrix 210, are arranged in the respective order. It can be saved as 0222343 as follows. The index pointer vector 213 may record the order of data as a range. For example, since the data of the first row exists within the range of [0:2], it can be seen that the number of data is 2, and the call indicates that there are the first two in the data vector 211, that is, 8 and 2. can be cut The next row is 5 indicated by the range of [2:3], and since there is no non-zero value in the next row, it can be written as [3:3].

2(b) shows a CSC scheme as matrix multiplication according to embodiments.

It may be matrix multiplication using a Compressed Sparse Column (CSC) method according to embodiments. At this time, the CSC method is performed in the same way as the CSR method, but data can be recorded centering on columns rather than rows. For example, as shown in (b) of FIG. 2, in the existing matrix 220, non-zero values can be stored in the data vector 221 in order from top to bottom from the leftmost column. there is. That is, the data vector 221 can store 8257192 values. The index vector 222 may sequentially store position values of rows in which non-zero values are located in the existing matrix 220 . For example, rows in which non-zero values included in the existing matrix 220 are located may be sequentially stored as 0014464. The index pointer vector 223 may record the order of data as a range, and may have, for example, a value of 011467.

However, although the expression method using a compressed sparse matrix is effective for sparse matrices, matrices in deep learning are often not sparse, and thus are not effective in terms of data storage space or computational complexity. Therefore, a calculation method capable of expressing various values in a neural network is required.

3 illustrates an example of performing matrix multiplication using bit vectors according to embodiments.

3 illustrates an example of performing matrix multiplication using a compressed bit vector according to embodiments.

A matrix multiplication method using a compressed bit vector according to embodiments may include converting data 301 into a bit vector 302 and converting the bit vector 302 into a compressed bit vector 303 .

The bit vector 302 according to the embodiments generates bit vectors as many as the distinct values of the data 301, and stores the position of a specific value among the data by expressing it as 1 in the bit vector representing the corresponding value. can For example, in the case of 1, which is the first value included in the data 301, it can be expressed as the first 1 of the bit vector 302 for 1. In addition, in the case of 3, which is the second value included in the data 301, it can be expressed as 1 in the second position of the bit vector 302 for 3.

The compressed bit vector 303 according to the embodiments is compressed to represent repetitive 0s or 1s as a small value, and a Run Length Encoding (RLE) method may be used. Specifically, the length of the run may be recorded for the same value repeatedly appearing, for example, as shown in FIG. 3, it may be recorded that 8*0 appears twice.

However, the expression method using compressed bit vectors has a problem in that a large number of bit vectors are required to express matrix values in deep learning having various values as bit vectors must exist as many as the number of unique values existing in data. . In addition, the expression method using a compressed bit vector has a problem in that an operation cannot be performed immediately and compression must be released. Therefore, an operation method capable of performing a matrix multiplication operation more effectively in a neural network is required.

4 is a flowchart of a multidimensional matrix multiplication method according to embodiments.

As shown in FIG. 4 , the multidimensional matrix multiplication method according to the embodiments may include outputting a plurality of input matrices as a plurality of binarized matrices (S401).

A plurality of input matrices according to embodiments are matrices to be calculated, and may include input matrices for performing matrix multiplication. A plurality of input matrices may include multidimensional matrices. A plurality of input matrices may have, for example, a 3-dimensional matrix. A plurality of input matrices may be output as binarized matrices through quantization. At this time, quantization is a value configured to have a smaller bit length than the element value of the original matrix, for example, while learning to be as similar as or equal to the accuracy of the neural network model in the case of using the original value as much as possible, the original value It may be a task to express with a small bit length. The plurality of input matrices may include a first matrix and a second matrix.

According to embodiments, the binarized matrix may be a matrix in which values included in the input matrix are binarized through quantization or a matrix composed of values decomposable into binarized values. The binarized matrix has the same appearance as the input matrix, but values included in the binarized matrix may have binarized values.

As shown in FIG. 1 , the multidimensional matrix multiplication method according to the embodiments may include compressing a plurality of binarized matrices and converting them into a plurality of compressed bit vectors (S402).

According to embodiments, the binarized matrix may be converted into a one-dimensional bit vector in an unfolded state. The converted bit vector may be compressed through a run length encoding (RLE) method. Specifically, the converted bit vector may be compressed by recording the values of bits included in the converted bit vector.

According to embodiments, the compressed bit vector may include 0, 1, and the number of repetitions of 0 or 1 as bits. Specifically, the value of the compressed bit vector may include whether the vector is compressed (0 or 1), values of 0 and 1, and the number of repetitions of 0 and 1. Accordingly, the multidimensional matrix multiplication method according to the embodiments can greatly reduce the size of an input matrix through a compressed bit vector.

As shown in FIG. 4, the multidimensional matrix multiplication method according to the embodiments may include calculating a bit at a specific position on a plurality of compressed bit vectors (S403).

In the compressed bit vector according to embodiments, the position of a specific bit existing in the compressed bit vector may be determined by a cursor, or a bit value at a specific position may be known. In this case, the cursor is a virtual cursor and may be used to reduce positional information of a specific bit of the compressed bit vector to positional information of a specific bit of the uncompressed bit vector. At this time, the cursor may obtain a location in a currently compressed word unit and a visit location in a compressed form corresponding to the location in order to obtain location information of a specific bit for the uncompressed bit vector. .

In the multidimensional matrix multiplication method according to the embodiments, when 0 is included in a size unit in which the size of a bit is divided, a region including 0 is skipped when examining through a cursor, thereby reducing processing speed. . Specifically, in the multidimensional matrix multiplication method according to the embodiments, a bit visit unit may be skipped through a cursor until a bit including 1 appears, or the cursor may be moved to a position of a specific bit.

As shown in FIG. 4 , the multidimensional matrix multiplication method according to the embodiments may include performing multiplication between a plurality of compressed bit vectors and outputting a resulting matrix (S404). In this case, an XNOR operator and a bitcount operation for calculating the number of bits having a value of 1 may be included to perform multiplication between compressed bit vectors.

According to embodiments, compressed bit vectors may be logically grouped for calculation. Specifically, the bits included in the compressed bit vector may be grouped in logical block units. The compressed bit vector may be grouped in units of logical blocks including the number of bits corresponding to the number of columns of the first matrix as the input matrix. In addition, the compressed bit vector may be grouped in units of logical blocks including the number of bits corresponding to the number of rows of the second matrix, which is the input matrix.

According to embodiments, it may be determined whether the grouped bits are matching or mismatching with each other by a preset operation. Specifically, it may be determined whether the plurality of grouped bits, which are the target of operation, are matched or mismatched with each other through a predetermined operation. In this case, the preset operation may be exclusive NOR (XNOR, exclusive Nor). That is, the grouped bits may calculate the number of 1s through a preset operation, for example, an XNOR operation using the bitcount function.

In the resulting matrix output according to embodiments, a bit position may be set based on the calculated number of 1's. Specifically, in the output result matrix, the position where the value of each bit is input may be set based on the number of 1's.

That is, the multidimensional matrix multiplication method according to the embodiments may perform multiplication between compressed bit vectors in an uncompressed state, that is, in a state in which bit vectors are compressed, and output a resulting matrix.

As shown in FIG. 4 , the multidimensional matrix multiplication method according to the embodiments may include transforming a resulting matrix corresponding to a position of a specific bit on a compressed bit vector (S405).

According to embodiments, the output result matrix may be converted into an uncompressed result matrix based on positional information of a specific bit with respect to an uncompressed bit vector obtained by a cursor.

In this case, according to embodiments, it may be determined whether logically grouped bits match or mismatch each other by a preset operation. Specifically, it may be determined whether the plurality of grouped bits, which are the target of operation, are matched or mismatched with each other through a predetermined operation. In this case, the preset operation may be XNOR. That is, the grouped bits may calculate the number of 1s through a preset operation, for example, an XNOR operation using the bitcount function. Based on the calculated number of 1's, the position of the bit may be set. Specifically, in the result matrix to be output, the position where the value of each bit is input may be set based on the number of 1's.

In the multidimensional matrix multiplication method according to the embodiments, when the operation is completed on the bits belonging to the first group of the compressed bit vector, the operation may be repeatedly performed on the bits belonging to the second group. In this case, the second group may perform step s403.

In the multidimensional matrix multiplication method according to the embodiments, when all of the steps s401 to s403 are performed for all block units (including the first group and the second group) divided into groups according to the embodiments, block unit Step s404 may be performed by circularly shifting the bit vector to the left by the size.

Hereinafter, outputting an input matrix as a binarized matrix will be described in detail.

5 is a diagram schematically showing that binarization of an input matrix and matrix multiplication thereof according to embodiments can be approximated with a result of matrix multiplication of existing input matrices as they are.

As shown in FIG. 5, the input matrix 510 (eg, the input layer described in FIG. 1, the data described in FIGS. 2 and 3, and the input matrix described in FIG. 4) is a plurality of multi-dimensional objects to be calculated. Can contain matrices or tensors. In FIG. 5, the input matrix 510 is shown in three dimensions, but is not limited thereto, and any dimension may be supported. The input matrix 510 may include, for example, a first matrix I and a second matrix W.

As shown in FIG. 5 , values included in the input matrix 510 may be binarized and output as a binarized matrix 520 (eg, the binarized matrix described in FIG. 4 ). Specifically, values included in the input matrix 510 may be binarized after quantization, or may be output as a binarized matrix 520 composed of values decomposable into the binarized values. In this case, the identification number 521 may be a matrix (K) or a constant (α) for returning to a value before decomposition when the input matrix can be decomposed into a binarized value. For example, the value 0.2 of the first matrix (I) included in the input matrix 510 becomes 1 while being binarized, and the value -5 of the second matrix (I) included in the input matrix 510 is binarized - can be 1 In this case, the binarized matrix 520 may have a bit length smaller than that of the input matrix 510 and may include bits that may have the same or similar values as the input matrix 510 . The binarized matrix 520 may be converted into a bit vector.

Hereinafter, the process of compressing and converting a binary matrix converted into a bit vector into a compressed bit vector will be described in detail.

6 is a diagram schematically illustrating compression of a bit vector generated by solving a matrix and a tensor according to embodiments.

The multidimensional matrix multiplication method (eg, the matrix multiplication described in FIGS. 1 to 3 or the multidimensional matrix multiplication method described in FIGS. 4 to 5) according to the embodiments is a binarized matrix (eg, FIGS. 4 to 5). A step of expressing the binarized matrix described above as a compressed bit vector (eg, the compressed bit vector described in FIG. 4) (eg, s402 described in FIG. 4) may be included.

In (a) of FIG. 6, bit vectors obtained by unfolding binarized matrices and tensors are shown. An unfolded bit vector consists of, for example, 8,000 bits, in which case the bit vector can only have a value of 0 or 1. For convenience of description, the following assumes that one word is composed of 32 bits in an arithmetic device including a computing device, but is not limited thereto.

As shown in (b) of FIG. 6, since one word is 32 bits, grouping may be performed by 31 bits. For example, 8000 bits are grouped into 258 groups for 7998 bits, and 2 bits may remain. At this time, the 258 groups may be divided into a group including 1 and a group not including 1. For example, 2 groups may include 1, and 256 groups may not include 1. In this case, as shown in (b) of FIG. 6, a group including 1 and a group not including 1 may be separated and expressed as 31 bits and 256*31 bits.

As shown in (c) of FIG. 6, compression may not be performed on a group including 1. In this case, the first bit of the group may be represented as 0.

As shown in (c) of FIG. 6, compression may be performed on a group not including 1. In this case, the first bit of the group may be represented as 1. At this time, the number of repetitions of the group (256) (run-length) may be recorded in the remaining bits, for example, 100000000 may be recorded.

That is, as shown in (c) of FIG. 6, by performing compression according to whether each group includes 1, a compressed bit vector can be output.

Hereinafter, calculating the position of a specific bit on a compressed bit vector will be described in detail.

7 is a diagram schematically illustrating that a position of a specific bit is calculated by a cursor according to embodiments.

The multidimensional matrix multiplication method (eg, the matrix multiplication method described in FIGS. 1 to 3 and the multidimensional matrix multiplication method described in FIGS. 4 to 6) according to the embodiments is a cursor (eg, the cursor described in FIG. 4 or the virtual matrix multiplication method described in FIG. 4 ). cursor) to obtain location information of a specific bit (eg, s403 described in FIG. 4), and specifically, a compressed bit vector (eg, compression described in FIGS. 4 and 6). It is possible to obtain information for reducing position information of a specific bit to position information in an uncompressed state for a bit vector).

As shown in a) of FIG. 7 , the cursor according to the embodiments may skip visits in bit units until the next 1 bit appears. For example, it is possible to move from a unit bit including 1 to a unit bit including 1 by skipping a unit bit including only 0 among unit bits.

As shown in b) of FIG. 7, the cursor according to the embodiments may move the cursor to a position of a specific bit. For example, the cursor may move to a position of a specific bit having '1'.

Hereinafter, matrix multiplication using compressed bit vectors will be described in detail.

Figure 8 is a schematic diagram of performing matrix multiplication over a compressed bit vector according to embodiments.

The multidimensional matrix multiplication method according to the embodiments (eg, the matrix multiplication described in FIGS. 1 to 3 or the multidimensional matrix multiplication method described in FIGS. 4 to 6 ) is a result matrix 830 (eg, in FIG. 1 ). The step of outputting the described output layer and the result matrix described in FIG. 4 (eg, s404 to s405 described in FIG. 4 ) may be included. Specifically, the multidimensional matrix multiplication method converts the input matrix 810 (eg, the input layer described in FIG. 1, the data described in FIGS. 2 to 3, and the input matrix described in FIGS. 4 to 5) into an unfolded vector. 820 (eg, the unfolded matrix described in FIG. 6), and the unfolded vector can be output as a result matrix 830.

An input matrix 810 according to embodiments may include an input matrix I and a weight matrix W. The input matrix I and the weight matrix W may be converted into a matrix having a real value of 0 or 1 by the quantization technique described in FIGS. 4 and 5 . Values of 1 to 15 of the input matrices 810 shown in FIG. 8 represent positions of the values of the input matrices 810 as easy-to-understand numbers. That is, the input matrix 810 may be converted into a binarized matrix (eg, the binarized matrix described in FIGS. 4 to 6) through a quantization technique.

The binarized matrix according to the embodiments may be converted into an unfolded vector 820 through the unfolding process described in FIGS. 4 and 6 . Through the unfolding process, the vector values of the input vector I and the vector values of the weight vector W can be listed on the unfolded vectors, unfold(I) and unfold(W). At this time, numbers corresponding to unfold(I) and unfold(W), for example, 1 included in unfold(I) and 1 corresponding to unfold(W) may indicate values to be multiplied with each other.

At this time, in response to the fact that the distinction between the same row or the same column is lost in the unfolding process according to the embodiments, the multidimensional matrix multiplication method according to the embodiments has as many bits as the number of columns of the input matrix I or the number of rows of the weight matrix W. Grouping may be performed to group them in logical block units, and thus, grouped bits (eg, grouped bits described in FIG. 4) may be formed.

At this time, the multidimensional matrix multiplication method according to the embodiments may perform XNOR and / or bitcount within the grouped interval to obtain a result value, for example, Bitcount (XNOR (I, W)) can do.

As shown in FIG. 8, after outputting the resulting matrix 831 for the unfolded matrix 821, for the next interval in the vector through a left cyclic shift (for example, the cyclic shift described in FIG. 4) The operation can be repeated. That is, an operation may be performed on a matrix 822 represented by a bit vector, and a resulting matrix 832 may be output accordingly.

At this time, in the result matrix 830, result values may be sequentially written on ①, ②, ③, ④, ⑤, and ⑥. Through this, it is possible to perform efficient operations between multidimensional matrices by minimizing the number of operations applied in bit units.

In FIG. 8, for convenience of description, vectors are calculated through XNOR operation in an uncompressed state to indicate the position of a matrix element, but in actual processing, as shown in FIGS. 6 and 7, compressed bit vectors It is possible to perform the calculation shown in FIG. 8 for the target.

9 is a diagram schematically showing the configuration of a multidimensional matrix multiplication operation device according to embodiments.

The multidimensional matrix multiplication operation apparatus 900 according to embodiments may include a binarization module 910, a bit vector compression module 920, a cursor module 930, and a multiplication operation module 940.

The binarization module 910 according to embodiments includes a plurality of input matrices (eg, the input layer described in FIG. 1, the data described in FIGS. 2 to 3, and the input matrix described in FIGS. 4 to 5 and 8). A plurality of binarized matrices (eg, the binarized matrices described in FIGS. 4 to 6 and 8) may be output by performing quantization on . The binarization module 910 may perform, for example, s401 described in FIG. 4 .

The bit vector compression module 920 according to embodiments may convert a plurality of binarized matrices into a plurality of compressed bit vectors (eg, the compressed bit vectors described in FIGS. 4 and 6 to 7). there is. The bit vector compression module 920 may perform, for example, s402 described with reference to FIG. 4 . Specifically, the bit vector compression module 920 may unfold a plurality of binarized matrices, convert them into bit vectors, record values of bits included in the converted bit vectors, and perform compression of the converted bit vectors. . In this case, the value of the compressed bit vector may include 0, 1, and the number of repetitions of 0 and 1.

The cursor module 930 according to embodiments may calculate a position of a specific bit on a plurality of compressed bit vectors or calculate a bit value of a specific position. The cursor module 930 may acquire a specific bit (eg, a position of 1) using a cursor (eg, the cursor described in FIGS. 4 to 5 or a virtual cursor). Cursor module 930 ) may perform, for example, s403 described in Figure 4. The cursor module 930 may also skip calculation so that other 0 values are not checked until the next 1-bit value as shown in Figure 7. can run

The multiplication operation module 940 according to embodiments may perform multiplication of a plurality of compressed bit vectors and output a result matrix (eg, the result matrix described in FIGS. 4 and 8 ). The multiplication operation module 940 may perform, for example, s404 and s405 described in FIG. 4 . Specifically, the multiplication operation module 940 may group bits included in the bit vector compressed by at least one of the number of columns and the number of rows included in the plurality of input matrices in block units. The multiplication operation module 940 may perform a preset operation on the grouped bits. In this case, the preset operation may be exclusive NOR (XNOR, exclusive Nor). That is, the multiplication operation module 940 may calculate the number of 1s included in the grouped bits through a preset operation, for example, an XNOR operation using the bitcount function. The multiplication operation module 940 may output a result matrix by setting the number of 1's as positions. When the result matrix for the first group is output, the multiplication operation module 940 may perform multiplication between compressed bit vectors for the next group. When multiplication is performed on all of the grouped bits, the multiplication operation module 940 cyclically shifts the bit vector included in the bit vector compressed by the block unit size (for example, as described in FIGS. 4 and 8 ). cyclic shift) to perform multiplication of compressed bit vectors.

A storage device including a multidimensional matrix multiplication method, an arithmetic unit, and a program according to embodiments uses a matrix multiplication speedup technique using a compressed bit vector in low-performance PCs and embedded and mobile devices that do not have high-performance computing devices such as GPUs. The running model can be processed efficiently.

Terms such as first, second, etc. may be used to describe various components of the embodiments. However, interpretation of various components according to embodiments should not be limited by the above terms. These terms are only used to distinguish one component from another. only thing For example, a first user input signal may be referred to as a second user input signal. Similarly, the second user input signal may be referred to as the first user input signal. Use of these terms should be construed as not departing from the scope of the various embodiments. Although both the first user input signal and the second user input signal are user input signals, they do not mean the same user input signals unless the context clearly indicates otherwise.

Terms used to describe the embodiments are used for the purpose of describing specific embodiments and are not intended to limit the embodiments. As used in the description of the embodiments and in the claims, the singular is intended to include the plural unless the context clearly dictates otherwise. and/or expressions are used in a sense that includes all possible combinations between the terms. The expression includes describes that there are features, numbers, steps, elements, and/or components, and does not imply that additional features, numbers, steps, elements, and/or components are not included. . Conditional expressions such as when ~, when, etc., used to describe the embodiments, are not limited to optional cases. When a specific condition is satisfied, a related action is performed in response to the specific condition, or a related definition is intended to be interpreted.

The above description is merely an example of the technical idea, and those skilled in the art to which the embodiments belong will be able to make various modifications and variations without departing from the essential characteristics of the embodiments.

Therefore, the embodiments disclosed above are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea is not limited by these embodiments of the present invention.

The protection scope of the present invention should be construed according to the following claims, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

810: input matrix
820: Unfolded vector
830: result matrix

Claims (15)

performing quantization on a plurality of input matrices and outputting a plurality of binarized matrices;
compressing the plurality of binarized matrices and converting them into a plurality of compressed bit vectors;
Based on whether or not each word constituting the bit vector is compressed on the plurality of compressed bit vectors, the position of the word currently visited by the cursor, the position of the bit indicated by the cursor in the word visited by the current cursor, restoring positional information of a specific bit through a cursor including the number of bits in a remaining word of the word;
outputting a resulting matrix by performing multiplication between the plurality of compressed bit vectors; and
transforming the resulting matrix in correspondence with positional information of the specific bit on the compressed bit vector; including,
Multidimensional matrix multiplication method. According to claim 1,
The step of converting into the plurality of compressed bit vectors,
converting the plurality of binarized matrices into bit vectors by unfolding;
performing compression of the converted bit vector by recording values of bits included in the converted bit vector; including,
Multidimensional matrix multiplication method. According to claim 1,
The compressed bit vector is represented by 0 and 1,
Multidimensional matrix multiplication method. According to claim 1,
The plurality of input matrices include a first matrix and a second matrix,
The step of outputting the result matrix,
grouping bits included in the compressed bit vector by the number of at least one of the number of columns of the first matrix and the number of rows of the second matrix in a logical block unit;
performing a predetermined operation on the grouped bits;
calculating the number of 1s included in the new bit vector obtained through the predetermined operation; and
outputting the resulting matrix by setting the number of 1's as a position; further comprising
Multidimensional matrix multiplication method. According to claim 4,
The preset operation is XNOR (exclusive nor) operation and Bitcount operation,
Multidimensional matrix multiplication method. According to claim 4,
The bits grouped in units of the logical block include a first group and a second group,
The step of outputting the result matrix,
outputting a result matrix for the second group as the result matrix for the first group is output; containing
Multidimensional matrix multiplication method. According to claim 6,
When the result matrix is output for all of the bits grouped in units of the logical block,
circularly shifting a bit vector included in the compressed bit vector by the size of the logical block unit;
grouping bits included in the compressed bit vector by the number of at least one of the number of columns of the first matrix and the number of rows of the second matrix in logical block units;
performing a predetermined operation on the grouped bits;
calculating the number of 1s included in the new bit vector obtained through the predetermined operation; and
outputting the resulting matrix by setting the number of 1's as a position; further comprising
Multidimensional matrix multiplication method. a binarization module that performs quantization on a plurality of input matrices and outputs a plurality of binarized matrices;
a bit vector compression module for converting the plurality of binarized matrices into a plurality of compressed bit vectors;
Based on whether or not each word constituting the bit vector is compressed on the plurality of compressed bit vectors, the position of the word currently visited by the cursor, the position of the bit indicated by the cursor in the word visited by the current cursor, a cursor module that restores positional information of a specific bit through a cursor including the number of bits in the remaining word of the word; and
a multiplication operation module for outputting a resultant matrix by performing multiplication of the plurality of compressed bit vectors;
including,
Multidimensional matrix multiplication unit. According to claim 8,
The bit vector compression module,
Unfolding the plurality of binarized matrices and converting them into bit vectors;
Recording the values of bits included in the converted bit vector to perform compression of the converted bit vector,
Multidimensional matrix multiplication unit. According to claim 8,
The bit vector is represented by 0 and 1,
Multidimensional matrix multiplication unit. According to claim 8,
The plurality of input matrices include a first matrix and a second matrix,
The multiplication operation module,
Grouping the bits included in the compressed bit vector by the number of at least one of the number of columns of the first matrix or the number of rows of the second matrix in logical block units,
Performing a predetermined operation on the grouped bits;
Calculate the number of 1s included in the new bit vector obtained through the preset operation,
Outputting the result matrix by setting the number of 1s as a position,
Multidimensional matrix multiplication unit. According to claim 11,
The preset operation is an XNOR operation and a Bitcount operation,
Multidimensional matrix multiplication unit. According to claim 11,
The bits grouped in units of the logical block include a first group and a second group,
The multiplication operation module,
Outputting a result matrix for the second group as the result matrix is output for the first group,
Multidimensional matrix multiplication unit. According to claim 13,
When the multiplication operation module outputs a result matrix for all of the bits grouped in units of the logical block,
The multiplication operation module,
Cyclically shifting a bit vector included in the compressed bit vector by the size of the logical block unit,
Grouping the bits included in the compressed bit vector by the number of at least one of the number of columns of the first matrix or the number of rows of the second matrix in logical block units,
Performing a predetermined operation on the grouped bits;
Calculate the number of 1s included in the new bit vector obtained through the preset operation,
Outputting the result matrix by setting the number of 1s as a position,
Multidimensional matrix multiplication unit. quantization is performed on a plurality of input matrices and output as a plurality of binarized matrices;
converting the plurality of binarized matrices into a plurality of compressed bit vectors;
Based on whether or not each word constituting the bit vector is compressed on the plurality of compressed bit vectors, the position of the word currently visited by the cursor, the position of the bit indicated by the cursor in the word visited by the current cursor, returning positional information of a specific bit through a cursor including the number of bits in the remaining word of the word;
outputting a resulting matrix by performing multiplication between the plurality of compressed bit vectors;
transforming the resulting matrix in response to positional information of the specific bit on the compressed bit vector;
A storage medium that stores a multidimensional matrix multiplication program.