KR101929847B1 - Apparatus and method for computing a sparse matrix - Google Patents

Abstract

The present invention relates to an apparatus for calculating a sparse matrix which can ultimately increase the speed of matrix calculation and a method thereof. The apparatus for calculating a sparse matrix comprises: a memory to store sparse matrices in which information indicating the number of continuous zeros in an identical row or column directions is represented by mantissa bits of the number of floating points representing values of elements of matrices; and one or more matrix calculation units to calculate sparse matrices accessed in the memory. A matrix calculation unit includes: a multiplication and accumulator (MAC) to matrix-multiply values of elements sequentially inputted in different paths; and a controller to sequentially select values of elements making up each sparse matrix accessed in the memory to output the values to a multiplier of the MAC, and output values to make an output of the multiplier, a zero, without selecting the number of values of elements indicated by the run length encoding when mantissa bits of the number of floating points for selected elements are run length encoded.

Description

[0001] APPARATUS AND METHOD FOR COMPUTING A SPARSE MATRIX [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to matrix operations, and more particularly, to an apparatus and method for computing a sparse matrix.

FIG. 1 is a graphical representation of a GEMM (General Matrix Multiplication), and FIG. 2 is a schematic diagram of a general matrix multiplication method.

Matrix operation plays a key role in science-related programs, image processing programs, and artificial intelligence (deep learning) programs, and mainly uses GEMM (General Matrix Multiplication) routines of BLAS (Basic Linear Algebra Subprograms) .

The core of the GEMM is to multiply the matrix A by the matrix B to calculate the matrix C. As shown in FIG. 2, the row of the matrix A is multiplied by the elements of the column of the matrix B, Compute the elements of matrix C.

The calculation of FIG. 2 can be expressed by the following equation.

These calculations are calculated by the processor through the FOR loop as follows. This requires a very large number of memory references, multiplication and addition when the size of matrix A is M × K and the size of matrix B is K × N.

If the matrix multiplication is performed by hardware rather than a program, the MAC (multiplication and accumulator) shown in FIG. 3 can be utilized.

On the other hand, a sparse matrix has a large number of elements of '0' among elements of the matrix, and methods for reducing memory space for storing the matrix have been proposed and used. A typical method is a method using a key and a method using a linked list.

The matrix value (element value) used for the matrix multiplication is usually a floating-point number, and the method of expressing the floating-point number is classified according to the number of bits as shown in FIG. 4, It is.

In FIG. 4, S denotes the quantity and the sign of the mantissa, E is the exponent and M is the mantissa. These floating point numbers have the following special expressions:

S = 0, E = all 1, M = all 0 → Infinity.

S = 0, E = all 1, M = not all 0 → + NaN.

S = 1, E = all 1, M = all 0 - > -Infinity.

S = 1, E = all 1, M = not all 0 -NaN.

As described above, in performing matrix multiplication with hardware, there is a restriction in improving the operation speed, in particular, in performing multiplication for a sparse matrix because unnecessary operations and memory references are repeated.

Korean Registered Patent No. 10-1400577 Korean Patent Publication No. 10-2017-0089389

Accordingly, it is an object of the present invention to solve the above-mentioned problems, and it is a primary object of the present invention to provide a method and apparatus for reducing the number of unnecessary operations on a sparse matrix and a reference number of memory or matrix data, And to provide a method and an apparatus for operating a sparse matrix that can be improved.

According to an aspect of the present invention, there is provided a method for calculating a sparse matrix,

Accessing in the memory sparse matrices in which information indicating the number of consecutive "zeros" in the same row or column direction is represented by mantissa bits of a floating-point number;

And sequentially outputting the values of the elements constituting each accessed sparse matrix to the matrix operation unit, wherein the mantissa bits of the floating-point number expressing the value of the selected element represent the number of consecutive zeros And outputting 'zero' without selecting a value of an element by the number of consecutive 'zeros' when the information is information,

Further, prior to accessing the sparse matrix in the memory,

And a value of each element is represented by a floating-point number, and the element is stored in the memory, and the value of each element constituting the sparse matrix is consecutive in the same row direction or column direction, end,

Length encoding a number of consecutive zeroes and converting the number of consecutive zeroes into a value of an element expressed by mantissa bits of a floating-point number, and storing the transformed value.

Further, in the sparse matrix computation method of the above-described embodiment, the sparse matrix accessed in the memory divides the sparse matrix stored in the memory into units of mxn (m, n is a natural number) .

The value of an element having a value of " zero " continuously in the same row direction or column direction among the values of elements constituting the sparse matrix,

(S) bit indicating row or column priority, an exponent (E) bit having both "1" and mantissa (M) bits indicating the number of consecutive zeros in a row or column, And the number of 'zero' is run-length encoded.

According to another aspect of the present invention, there is provided a sparse matrix computing apparatus comprising:

A memory for storing sparse matrices in which information indicating the number of 'zeroes' continuous in the same row direction or column direction is represented by mantissa bits of a floating-point number representing the value of each element constituting the matrix;

And at least one matrix operation unit for performing matrix operation on the accessed sparse matrices in the memory,

A MAC (Multiplication and Accumulator) for matrix operation of values of elements sequentially input in different paths;

The run length encoding value is calculated by sequentially selecting values of elements constituting each of the sparse matrices accessed in the memory and outputting the values to the multiplier of the MAC when the mantissa bits of the floating point number for the selected element are run- And outputs a value for making the output of the multiplier 'zero' without selecting a value of the number of elements to be indicated.

In the above-described sparse matrix arithmetic unit, information indicating the number of 'zeroes' continuous in the same row direction or column direction is expressed as a scarce number expressed by mantissa bits of a floating-point number representing the value of each element constituting the matrix And an external processor for generating a matrix and transmitting the matrix to the memory.

In the sparse matrix computing device having the above-described configuration, the value of an element having a value of " zero " continuously in the same row direction or column direction among the values of the elements constituting the sparse matrix,

(S) bit indicating row or column priority, an exponent (E) bit having both "1" and mantissa (M) bits indicating the number of consecutive zeros in a row or column, And the number of 'zero' is run-length encoded.

When a matrix operation is performed by a method or apparatus according to an embodiment of the present invention, matrix operation can be performed without referring to the value of an element as many as the number of consecutive elements in the same row direction or column direction when the number is zero Therefore, it is possible to reduce the number of reference times of the matrix data, thereby ultimately improving the speed of the matrix operation. Furthermore, the larger the size of the sparse matrix, the more the number of reference times of the matrix data increases, so that the effect of the present invention can be maximized.

In particular, mathematical operations in the fields of science and technology, image processing, and artificial intelligence (deep running) require a very large amount of computation. In performing such a matrix operation, it is possible to reduce the amount of matrix calculation by simply reducing the number of times that row and column data having a value of 'zero' are not accessed or referred to in the memory, There is an advantage to be able to do.

FIG. 1 is a schematic diagram of a GEMM (General Matrix Multiplication). FIG.
Figure 2 is an example of matrix multiplication.
3 is a diagram illustrating an example of a general MAC (Multiplier and Accumulator) configuration used for matrix operation.
4 is an exemplary representation of a floating-point number;
5 is a diagram illustrating an example of a peripheral configuration of a matrix computation apparatus according to an embodiment of the present invention;
FIG. 6 is a diagram illustrating a configuration example of a matrix computing device in FIG. 5; FIG.
7 and 8 are diagrams for explaining a method of representing a floating-point number according to an embodiment of the present invention.
9 is a diagram for explaining a process of dividing and accessing a sparse matrix according to an embodiment of the present invention.
10 and 11 are diagrams for further illustrating effects according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

For reference, the term 'element' used in the following description is defined as meaning an entry constituting a matrix, and 'element value' means a matrix value of the element defined above. do. Since the matrix value used in the normal matrix multiplication, that is, the value of an element is represented by a floating-point number, the value of the element is also represented by a floating-point number. On the other hand, the term 'follow-up' is defined as a meaning that includes 'itself in the same row direction or column direction, followed by' meaning. In the following, embodiments of the present invention will be described by exemplifying a sparse matrix multiplication during a sparse matrix operation.

FIG. 5 illustrates a peripheral configuration diagram of a matrix operation device 400 according to an embodiment of the present invention, and FIG. 6 illustrates a configuration diagram of a matrix operation device in FIG.

5, the external processor 100 is connected to a hardware (e.g., FPGA) board on which a matrix computing device 400 according to an embodiment of the present invention is mounted and a system bus 120 . The external processor 100 reads the sparse matrix data necessary for the calculation of the sparse matrix, for example, the multiplication of the sparse matrix, from the main memory 110 and transfers it to the hardware board.

The value of each element of the delivered sparse matrix is expressed by a floating-point number as described above. According to the implementation method, the external processor 100 merely stores the value of each element as a rare- Information indicating the number of 'zeros' consecutive in the same row direction or column direction according to the embodiment of the present invention is transmitted as a floating-point number mantissa representing the value of each element constituting the matrix (M) bits to be transmitted to the hardware board. That is, the external processor 100 that generates the sparse matrix in which the information indicating the number of 'zeroes' continuous in the same row direction or column direction in the matrix is expressed by the mantissa bits of the floating-point number, Can be regarded as a component of.

As shown in FIG. 5, a hardware board for distributing the load of the external processor 100 includes a bus I / F (Interface) 210, an FPGA internal system bus 220, a memory 300, (400) can be mounted. 5, the memory 300 and the matrix calculator 400 are separated from each other. However, the memory 300 includes one or more matrix calculators constituting the matrix calculator 400 and one matrix calculator 400 .

In the memory 300, information indicating the number of 'zeros' consecutive in the same row direction or column direction is stored as a floating-point number of mantissa bits representing the value of each element constituting the matrix do. These sparse matrices may be delivered from the external processor 100.

When a sparse matrix represented by a floating-point number is received from the external processor 100 by a conventional method, the controller in the matrix calculator 400, which will be described later, Quot; zero " is represented by the mantissa bits of the floating-point number and may be used for the matrix operation.

On the other hand, the matrix calculator 400 includes one or more matrix calculators for performing matrix calculations on the sparse matrices accessed in the memory 300, and the matrix calculator includes, as an example,

A MAC (Multiplication and Accumulator) 430 for performing matrix multiplication on values of elements sequentially input in different paths,

And sequentially outputs the values of the elements constituting each of the sparse matrices accessed in the memory 300 to the multiplier of the MAC 430. When the mantissa of the floating point number of the selected element is a run length, And outputs a value for making the output of the multiplier 'zero' without selecting a value of an element as many as the number indicated by the run length encoded value when encoded.

The controller 420 is connectable to the system bus 220 via the bus I / F 410 and accesses the sparse matrix from the memory 300 in the DMA scheme and transfers the sparse matrix to the MAC 430 in the subsequent stage.

Each of the plurality of matrix operation units having the configuration as shown in FIG. 6 may be connected to the system bus 220 to process the matrix multiplication in parallel. To this end, the controller 420 may access the sparse matrix stored in the memory 300 by dividing the m × n (m, n is a natural number) unit. Of course, the natural numbers m and n should have values smaller than the lengths of the rows and columns of the stored sparse matrix. Since the MAC 430 shown in FIG. 6 has the configuration shown in FIG. 3, a detailed description thereof will be omitted.

Hereinafter, the operation of the matrix calculator 400 including the memory 300 and one or more matrix calculators (more specifically, a matrix multiplier) will be described in detail with reference to FIGS. 7 to 11. FIG.

7 and 8 are diagrams for explaining a floating-point number representation method according to an embodiment of the present invention. More specifically, FIG. 7 shows an example of applying a row-first run length encoding (RLE) And FIG. 8 shows an example in the case of applying the column-priority RLE.

In the embodiment of the present invention, NaN (Not a Number), which is not used as an input value of a matrix operation, is used for RLE and expressed as follows.

RRLE: S = 0, E = all, M = number of zeroes → row-major run length encoding

CRLE: S = 1, E = all, M = number of 'zeroes' → column-major run length encoding

That is, the number of consecutive zeros in the row direction is displayed in M (mantissa bit) when S (sign bit) is '0' and E (exponent bit) is '1' in row priority. In the case of column priority, the number of consecutive zeros in the same column direction is indicated in M when S is '1' and E is all '1'.

In other words, assuming that FIG. 7 is a row-first matrix, if the value of the element is 'zero' successively from row 2 to column 7 in row 1, the values of the elements in column 1 row 2 are S = 0, E = all1, M = 5. At this time, if the mantissa M is run-length encoded, the value of the corresponding element can be represented by a sign decimal point number such as {0, 1111_1111,000_0000_0000_0000_0000_0101}. Since the values of the elements in row 1, row 3 and column 3 are S = 0, E = all 1 and M = 4, the value of the corresponding element (column 1, row 3) is {0,1111_1111 , 000_0000_0000_0000_0000_0100}. If there is one 'zero', it can be displayed as {0,0000_0000,000_0000_0000_0000_0000} or {0,1111_1111,000_0000_0000_0000_0000_0001} as the original expression.

In the row-first case, as in the case of the second and third rows in FIG. 7, all the RLEs are newly started in the row units.

As shown in FIG. 8, the values of the elements constituting the column-first matrix in the same manner as described above are converted such that information indicating the number of 'zeroes' continuous in the same column direction is represented by mantissa (M) bits of the floating- can do.

In the above-described method, the controller 420 constituting the external processor 100 or the matrix multiplier divides the information indicating the number of 'zeros' consecutive in the same row or column direction into mantissa (M) bits of the floating-point number The sparse matrix is transformed to be represented. The thus transformed sparse matrices are stored in the memory 300 and accessed by the controller 420 in one or more matrix multipliers described below to be used for matrix multiplication.

The controller 420 may be divided and accessed in the row direction or column direction according to the number of matrix multiplication units as shown in FIG.

For reference, FIGS. 10 and 11 show drawings for further illustrating effects according to the embodiment of the present invention.

The controller 420 accesses the sparse matrix necessary for the matrix multiplication in the memory 300. The controller 420 sequentially selects the values of the elements constituting the accessed sparse matrix and outputs them to the multiplier of the MAC (Multiplication and Accumulator) (M) bits of a floating-point number representing a value of a selected element are information indicating the number of 'zeroes' followed by the output of the multiplier without selecting the value of the element as many as the number of the following 'zeroes' And outputs a value for making it zero (for example, logic level zero).

10 and 11, when the sparse matrix A and the sparse matrix B are multiplied as shown in Fig. 10, when the calculation for the first row of the matrix A is performed, The multiplication calculation for the elements of a matrix B becomes unnecessary. This means that it is not necessary to refer to the values of the elements of the matrix B marked with the corresponding hatched.

When the controller 420 determines that the mantissa bit of the floating-point number representing the value of the element selected for output to the MAC 420 is the information indicating the number of 'zeroes' followed by the number of 'zero' So that the matrix multiplication is performed by outputting a value for making the output of the multiplier zero.

In this way, in the case of performing matrix multiplication of column 2 of matrix B in FIG. 10, it is not necessary to refer to values 1-4 of matrix A.

Therefore, when the matrix multiplication is performed by the method or apparatus according to the embodiment of the present invention, matrix multiplication can be performed without referring to the values of elements as many as the number of consecutive elements in the same row direction or column direction, , It is possible to reduce the number of matrix data references and ultimately improve the speed of matrix operation. Furthermore, the larger the size of the sparse matrix, the more the number of reference times of the matrix data increases, so that the effect of the present invention can be maximized.

In particular, mathematical operations in the fields of science and technology, image processing, and artificial intelligence (deep running) require a very large amount of computation. In performing such a matrix operation, it is possible to reduce the amount of matrix calculation by reducing the number of times that row and column data having a value of 'zero' are not accessed or referred to in the memory, thereby improving the calculation performance.

While the invention has been shown and described with reference to certain embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined by the appended claims. For example, although the matrix multiplication is exemplified in the embodiment of the present invention, matrix operation can be performed by applying the technical idea of the present invention to matrix addition or subtraction.

Claims (7)

Accessing in the memory sparse matrices in which information indicating the number of consecutive "zeros" in the same row or column direction is represented by mantissa bits of a floating-point number;
And sequentially outputting the values of the elements constituting each accessed sparse matrix to the matrix operation unit, wherein the mantissa bits of the floating-point number expressing the value of the selected element represent the number of consecutive zeros And outputting 'zero' to the matrix operation unit without selecting a value of an element as many as the number of consecutive 'zeros' when the information is information, wherein, before accessing the sparse matrix in the memory,
And a value of each element is represented by a floating-point number, and the element is stored in the memory, and the value of each element constituting the sparse matrix is consecutive in the same row direction or column direction, end,
Encoding the number of consecutive 'zeroes' followed by converting the number of successive zeroes into a value of an element, and storing the transformed value as a mantissa bit of a floating-point number. 2. The method of claim 1, wherein the sparse matrix accessed in the memory is a matrix obtained by dividing a sparse matrix stored in the memory in units of mxn (m, n is a natural number). [2] The method of claim 1, wherein the value of an element having a value of " zero " continuously in the same row direction or column direction among the values of elements constituting the sparse matrix,
(S) bit indicating row or column priority, an exponent (E) bit having both "1" and mantissa (M) bits indicating the number of consecutive zeros in a row or column, Wherein the number of 'zeroes' is run-length encoded. Accessing in the memory sparse matrices in which information indicating the number of consecutive "zeros" in the same row or column direction is represented by mantissa bits of a floating-point number;
And sequentially outputs the values of the elements constituting each accessed sparse matrix to the matrix operation unit, wherein the mantissa bits of the floating-point number expressing the value of the selected element represent the number of consecutive zeros And outputting 'zero' to the matrix operation unit without selecting a value of an element as many as the number of consecutive 'zeros' when the information is information,
The value of an element having a value of " zero " continuously in the same row direction or column direction among the values of elements constituting the sparse matrix,
(S) bit indicating row or column priority, an exponent (E) bit having both "1" and mantissa (M) bits indicating the number of consecutive zeros in a row or column, Wherein the number of 'zeroes' is run-length encoded. A memory for storing sparse matrices in which information indicating the number of 'zeroes' continuous in the same row direction or column direction is represented by mantissa bits of a floating-point number representing the value of each element constituting the matrix;
And at least one matrix operation unit for performing matrix operation on the accessed sparse matrices in the memory,
A MAC (Multiplication and Accumulator) for matrix multiplication of values of elements sequentially input in different paths;
The run length encoding value is calculated by sequentially selecting values of elements constituting each of the sparse matrices accessed in the memory and outputting the values to the multiplier of the MAC when the mantissa bits of the floating point number for the selected element are run- And outputs a value for making the output of the multiplier 'zero' without selecting a value of an element by a specified number of elements. The method according to claim 5, wherein a sparse matrix is generated in which information indicating the number of 'zero' continuous in the same row direction or column direction is expressed by mantissa bits of a floating-point number representing the value of each element constituting the matrix And an external processor for transferring the sparse matrix to the memory. The method according to claim 5 or 6, wherein values of elements having a value of " zero " continuously in the same row direction or column direction among the values of the elements constituting the sparse matrix,
(S) bit indicating row or column priority, an exponent (E) bit having both "1" and mantissa (M) bits indicating the number of consecutive zeros in a row or column, Wherein the number of 'zeroes' is run-length encoded.

Images